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We prove a many-server heavy-traffic fluid limit for an overloaded Markovian queueing system having two customer 
classes and two service pools, known in the call-center literature as the X model. The system uses the fixed-queue- 
ratio-with-thresholds (FQR-T) control, which we proposed in a recent paper as a way for one service system to 
help another in face of an unexpected overload. Under FQR-T, customers are served by their own service pool 
until a threshold is exceeded. Then, one-way sharing is activated with customers from one class allowed to be 
served in both pools. After the control is activated, it aims to keep the two queues at a pre-specified fixed 
ratio. For large systems that fixed ratio is achieved approximately. For the fluid limit, or FWLLN, we consider 
a sequence of properly scaled X models in overload operating under FQR-T. Our proof of the FWLLN follows 
the compactness approach, i.e., we show that the sequence of scaled processes is tight, and then show that all 
converging subsequences have the specified limit. The characterization step is complicated because the queue- 
difference processes, which determine the customer-server assignments, need to be considered without spatial 
scaling. Asymptotically, these queue-difference processes operate in a faster time scale than the fluid-scaled 
processes. In the limit, due to a separation of time scales, the driving processes converge to a time-dependent 
steady state (or local average) of a time- varying fast-time-scale process (FTSP). This averaging principle allows 
us to replace the driving processes with the long-run average behavior of the FTSP. 

Key words: many-server queues; averaging principle; separation of time scales; state-space collapse; heavy-traffic 
fluid limit; overload control 

MSC2000 Subject Classification: Primary: 60F17, 60K25 ; Secondary: 60G70, 90B22 

OR/MS subject classification: Primary: Queues ; Secondary: Limit Theorems, Transient Results 



1. Introduction. In this paper we prove that the deterministic fluid approximation for the over- 
loaded X call-center model, suggested in [33] and analyzed in [35], arises as the many-server heavy-traffic 
fluid limit of a properly scaled sequence of overloaded Markovian X models under the fixed- queue-ratio- 
with-thresholds (FQR-T) control. The X model has two classes of customers and two service pools, one 
for each class, but with both pools capable of handling customers from either class. The service-time 
distributions depend on both the class and the pool. The FQR-T control was suggested in [33J as a way 
to automatically initiate sharing (i.e., sending customers from one class to the other service pool) when 
the system encounters an unexpected overload, while ensuring that sharing does not take place when it 
is not needed. 



1.1 A Series of Papers. This paper is the fourth in a series. First, in [33] we heuristically derived a 
stationary fluid approximation, whose purpose was to approximate the steady-state of a large many-server 
X system operating under FQR-T during the overload incident. More specifically, in [33] we assumed 
that a convex holding cost is incurred on both queues whenever the system is overloaded, and our aim 
was to develop a control designed to minimize that cost. We further assumed that the system becomes 
overloaded due to a sudden, unexpected shift in the arrival rates, with new levels that may not be known 
to the system managers, and that the staffing of the service pools cannot be changed quickly enough to 
respond to that sudden overload. Under the heuristic stationary fluid approximation, it was shown that 
FQR-T outperforms the fluid-optimal static (fixed numbers of servers) allocation, even when the new 
arrival rates are known to the system manager. 

Second, in [34] we applied a heavy-traffic averaging principle (AP) as an engineering principle to 
describe the transient (time-dependent) behavior of a large overloaded X system operating under FQR- 
T. The suggested fluid approximation was expressed via an ordinary differential equation (ODE), which is 
driven by a stochastic process. Specifically, the expression of the fluid ODE as a function of time involves 
the local steady state of a stochastic process at each time point t > 0, which we named the fast-time-scale 
process (FTSP). As the name suggests, the FTSP operates in (an infinitely) faster time scale than the 
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processes approximated by the ODE, thus converges to its local steady state instantaneously at every 
time t > 0. Extensive simulation experiments showed that our approximations work remarkably well, 
even for surprisingly small systems, having as few as 25 servers in each pool. 

Third, in [35] we investigated the ODE suggested in [31] using a dynamical-system approach. The 
dynamical-system framework could not be applied directly, since the ODE is driven by a stochastic 
process, and its state space depends on the distributional characteristics of the FTSP. Nevertheless, we 
showed the a unique solution to the ODE exists over the positive halfline [0, oo). The stationary fluid 
approximation, derived heuristically in [33] . was shown to exist as the unique fixed point (or stationary 
point) for the fluid approximation. Moreover, we proved that the solution to the ODE converges to this 
stationary point, with the convergence being exponentially fast. In addition, a numerical algorithm to 
solve the ODE was developed, based on a combination of a matrix-geometric algorithm and the classical 
forward Euler method for solving ODE's. 

1.2 Overview. In this fourth paper, we will prove that the solution to the ODE in [331 [SS] is indeed 
the many-server heavy-traffic fluid limit of the overloaded X model, which we also call a functional weak 
law of large numbers (FWLLN); see Theorem 14. II see £13.41 for the key assumptions. In doing so, we will 
prove a strong version of state-space collapse (SSC) for server-assignment processes; see Corollarv l6.2l and 
Theorem 16.41 We will also prove a strong SSC result for the two-dimensional queue process in Corollary 
14.21 In a subsequent paper {36] we prove a functional central limit theorem (FCLT) refinement of the 
FWLLN here, which describes the stochastic fluctuations about the fluid path. 

We only consider the X model during the overload incident, once sharing has begun; that will be 
captured by our main assumptions, Assumptions [3TT1 and [3~2l As a consequence, the model is stationary 
but the evolution is transient. Because of customer abandonment, the stochastic models will all be stable, 
approaching proper steady-state distributions. As a further consequence, during the overload incident 
sharing will occur in only one direction, so that the overloaded X model actually behaves as an overloaded 
N model, but that requires proof; that follows from Corollary 16.31 and Theorem l6.4l 

Convergence to the fluid limit will be established in roughly three steps: (i) representing the sequence 
of systems f? £|5.1l and 16), (ii) proving that the sequence considered is C-tight f £|5.2[) , and (Hi) uniquely 
characterizing the limit ( [35] and $9]). The first representation step in £15.11 starts out in the usual 
way, involving rate-1 Poisson processes, as reviewed in [35] ■ However, the SSC part in SJBJ requires a 
delicate analysis of the unsealed sequence. The second tightness step in £15.21 is routine, but the final 
characterization step is challenging. These last two steps are part of the standard compactness approach 
to proving stochastic-process limits; see [6], [11], [32] and §11.6 in [45]. As reviewed in [11] and [32] . 
uniquely characterizing the limit is usually the most challenging part of the proof, but it is especially 
so here. Characterizing the limit is difficult because the FQR-T control is driven by a queue-difference 
process which is not being scaled and hence does not converge to a deterministic quantity with spatial 
scaling. However, the driving process operates in a different time scale than the fluid-scaled processes, 
asymptotically achieving a (time-dependent) steady state at each instant of time, yielding the AP. 

As was shown in [35], the AP and the FTSP also complicate the analysis of the limiting ODE. First, it 
requires that the steady state of a continuous-time Markov chain (CTMC), whose distribution depends 
on the solution to the ODE, be computed at every instant of time. (As explained in [35], this argument 
may seem circular at first, since the distribution of the FTSP is determined by the solution to the ODE, 
while the evolution of the solution to the ODE is determined by the behavior of the FTSP. However, the 
separation of time scales explains why this construction is consistent.) The second complication is that 
the AP produces a singularity region in the state space, causing the ODE to be discontinuous in its full 
state space. Hence, both the convergence to the many-server heavy-traffic fluid limit, and the analysis of 
the solution to the ODE depend heavily on the state space of the ODE, which is characterized in terms 
of the FTSP. For that reason, many of the results in [35J are needed for proving convergence. 

1.3 Literature. Our previous papers discuss related literature; see especially §1.2 of [33]. Our 
FQR-T control extends the FQR control suggested and studied in [T3J [TU [T5J , but the limits there were 
established for a different regime under different conditions. Here we propose FQR-T for overload control 
and establish limits for overloaded systems. Unlike that previous work, here the service rates may depend 
on both the customer class and the service pool in a very general way. In particular, our X model does 
not satisfy the conditions of the previous theorems even under normal loads. 
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There is a substantial literature on averaging principles; e.g., see [33] and references therein. However, 
there are only a few papers in the queueing literature involving averaging principles; see p. 71 of |45j for 
discussion. Two notable papers are Coffman et al. [5], which considers the diffusion limit of a polling 
system with zero switch-over times, and Hunt and Kurtz |17j . which considers large loss networks under 
a large family of controls. Reference [T7] is closely related to our work since it considers the fluid limits of 
such loss systems, with the control-driving process moving at a faster time scale than the other processes 
considered. For the important characterization step, we give two proofs, one in the main paper and the 
other in the appendix. The shorter proof in the main paper closely follows [17] , exploiting martingales and 
random measures, building on Kurtz [35]. In contrast, our second approach exploits stochastic bounds, 
which we also use in the important preliminary step establishing state space collapse. 

There is now a substantial literature on fluid limits for queueing models, some of which is reviewed in 
[45] . For recent work on many-server queues, see [19l [22] . Because of the separation of time scales here, 
our work is in the spirit of fluid limits for networks of many-server queues in [3J 0] , but again the specifics 
are quite different. Their separation of time scales justifies using a pointwise stationary approximation 
asymptotically, as in [211 HI] . 

2. Preliminaries. In this section we specify the queueing model, which we refer to as the X model. 
We then specify the FQR-T control. We then provide a short summary of the many-server heavy-traffic 
scaling and the different regimes. We conclude with our conventions about notation. 

2.1 The Original Queueing Model. The Markovian X model has two classes of customers, ini- 
tially arriving according to independent Poisson processes with rates Ai and A2. There are two queues, 
one for each class, in which customers that are not routed to service immediately upon arrival wait to 
be served. Customers are served from each queue in order of arrival. Each class-i customer has limited 
patience, which is assumed to be exponentially distributed with rate 6i, i — 1,2. If a customer does not 
enter service before he runs out of patience, then he abandons the queue. The abandonment keep the 
system stable for all arrival and service rates. 

There are two service pools, with pool j having m,j homogenous servers (or agents) working in parallel. 
This X model was introduced to study two large systems that are designed to operate independently under 
normal loads, but can help each other in face of unanticipated overloads. We assume that all servers are 
cross- trained, so that they can serve both classes. The service times depend on both the customer class i 
and the server type j, and are exponentially distributed; the mean service time for each class-i customer 
by each pool-j agent is All service times, abandonment times and arrival processes are assumed 

to be mutually independent. The FQR-T control described below assigns customers to servers. 

We assume that, at some unanticipated time, the arrival rates change instantaneously, with at least one 
increasing. At this time the overload incident has begun. We consider the system only after the overload 
incident has begun, assuming that it is in effect at the initial time 0. We further assume that the staffing 
cannot be changed (in the time scale under consideration) to respond to this unexpected change of arrival 
rates. Hence, the arrival processes change from Poisson with rates Ai and A2 to Poisson processes with 
rates Ai and A2, where A, < Wt//i,,i, i = 1,2 (normal loading), but Ai > jLtj^m, for at least one i (the 
unanticipated overload). (These new arrival rates may not be known by the system manager.) Without 
loss of generality, we assume that pool 1 (and class-1) is the overloaded (or more overloaded) pool. The 
fluid model (ODE) is an approximation for the system performance during the overload incident, so that 
we start with the new arrival rate pair (Ai, A2). (The overload control makes sense much more generally; 
we study its performance in this specific scenario.) 

The two service systems may be designed to operate independently under normal conditions (without 
any overload) for various reasons. In 33 ; , EH] we considered the common case in which there is no efficiency 
gain from service by cross-trained agents. Specifically, in [33) we assumed the strong inefficient sharing 
condition 

Mi.l > Mi, 2 and ^2,2 > j"2,i- (1) 
Under condition (TTJ), customers are served at a faster rate when served in their own service pool than 
when they are being served in the other-class pool. However, many results in |33) hold under the weaker 
basic inefficient sharing condition: /ii ! i/i2,2 > Mi, 2/42,1- 

It is easy to see that some sharing can be beneficial if one system is overloaded, while the other is 
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underloaded (has some slack), but sharing may not be desirable if both systems are overloaded. In order 
to motivate the need for sharing when both systems are overloaded, in |33) we considered a convex-cost 
framework. With that framework, in [33J we showed that sharing may be beneficial, even if it causes the 
total queue length (queue 1 plus queue 2) to increase. 

Let Qi(t) be the number of customers in the class- i queue at time i, and let Zij(t) be the number 
of class-i customers being served in pool j at time t, i,j — 1,2. Given a stationary (state-dependent) 
routing policy, the six-dimensional stochastic process 

X 6 (t) = (Q 1 (t),Q 2 (t),Z 1A (t),Z lt2 (t),Z 2A (t),Z 2}2 (t)), t>0, (2) 

becomes a six-dimensional CTMC. (= means equality by definition.) In principle, the optimal control 
could be found from the theory of Markov decision processes, but that approach seems prohibitively 
difficult. For a complete analysis, we would need to consider the unknown transient interval over which 
the overload occurs, and the random initial conditions, depending on the model parameters under normal 
loading. In summary, there is a genuine need for the simplifying approximation we develop. 

2.2 The FQR-T Control for the Original Queueing Model. The purpose of FQR-T is to 
prevent sharing when the system is not overloaded, and to rapidly start sharing when the arrival rates 
shift. If sharing is elected, then we allow sharing in only one direction. 

Assumption 2.1 (one-way sharing) Sharing is allowed in only one direction at any one time. 

When sharing takes place, FQR-T aims to keep the two queues at a certain ratio, depending on the 
direction of sharing. Thus, there is one ratio, r± t2 , which is the target ratio if class 1 is being helped 
by pool 2, and another target ratio, r 2l i, when class 2 is being helped by pool 1. As explained in [33], 
appropriate ratios can be found using the steady-state fluid approximation. In particular, the specific 
FQR-T control is optimal in the special case of a separable quadratic cost function. More generally, fixed 
ratios arc often approximately optimal. 

We now describe the control. The FQR-T control is based on two positive thresholds, ki l2 and k 2t \, 
and the two queue-ratio parameters, r± <2 and r 2t ±. We define two queue-difference stochastic processes 
D± t2 (t) = Qi(t)—ri t2 Q 2 (t) andZ?2.i = r 2j \Q 2 (t) — Q\(t). As shown in [33] . in that convex cost framework 
there is no incentive for sharing simultaneously in both directions. These ratio parameters should satisfy 
T\ 2 > J~2,i; see Proposition EC. 2 and (EC. 11) of [33]. hence we make the following assumption. 

Assumption 2.2 (ordered ratio parameters) The ratio parameters are assumed to satisfy n,2 > r 2 ,i- 

As long as Di^ 2 (t) < ki >2 and D 2j i(t) < k 2j i we consider the system to be normally loaded (i.e., not 
overloaded) so that no sharing is allowed. Hence, in that case, the two classes operate independently. 
Once one of these inequalities is violated, the system is considered to be overloaded, and sharing is 
initialized. For example, if D i:2 (t) > ki >2 and Z 2> i(t) — 0, then class 1 is judged to be overloaded and 
service-pool 2 is allowed to start helping queue 1: If a type-2 server becomes available at this time t, 
then it will take its next customer from the head of queue 1. When Di y2 (t) — k\^ 2 < 0, new sharing is 
not initiated. Then new sharing stops until one of the thresholds is next exceeded. However, sharing 
in the opposite direction with pool 1 servers helping class 2 is not allowed until both Z± <2 (t) — and 
D 2<1 (t) > k 2 ,x. 

It can be of interest to consider alternative variants of the FQR-T control just defined. For example, 
it may be desirable to relax the one-way sharing rule imposed above. We might use additional lower 
thresholds for Z\, 2 (t) and Z± t2 (t) to allow sharing to start more quickly in the opposite direction when 
the queue lengths indicate that is desirable. However, we do not discuss such control variants here. 

With the FQR-T control just defined, the six-dimensional stochastic process Xq = {Xe(t) : t > 0} in 
([2]) is a CTMC. (The control depends only on the process state.) This is a stationary model, but we are 
concerned with its transient behavior, because it is not starting in steady state. We aim to describe that 
transient behavior. The control keeps the two queues at approximately the target ratio, e.g., if queue 1 is 
being helped, then Q\(t) ss ri t2 Q 2 (t). If sharing is done in the opposite direction, then r 2f iQ 2 (t) w Qi(t) 
for all t > 0. That is substantiated by simulation experiments, some of which are reported in [33] [34]. In 
this paper we will prove that the s» signs are replaced with equality signs in the fluid limit. 
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2.3 Many-Server Heavy- Traffic Scaling. To develop the fluid limit, we consider a sequence of 
X systems, {Xg : n > 1} defined as in ([2]), indexed by n (denoted by superscript), using the standard 
many-server heavy-traffic scaling, i.e., with arrival rates and number of servers growing proportionally to 



Assumption 2.3 (many-server heavy-traffic scaling) As n — > oo, 

A™ m™ 

A™ = — > Xi and m% = — - > m», < X i ,m l < oo, (3) 

n n 

and the service and abandonment rates held fixed. 

We then define the associated fluid-scaled stochastic processes 

X2(t) = n-%"(i), t>0. (4) 
Where Xg is defined as in ([2]) for each n. 

For each system n, there are thresholds fc™ 2 and k 21 , scaled as suggested in [551 154] . 

Assumption 2.4 (scaled thresholds) For k\ t 2,^2,1 > and a sequence of positive numbers {c n : n > 1}, 
k™j/cn — > ki t j, = (1, 2) or (2, 1), where c„/n — > and Cn/y/n— > 00 as n — > 00. (5) 



The first scaling by n in Assumption 12.41 is chosen to make the thresholds asymptotically negligible 
in many-server heavy-traffic fluid scaling, so they have no asymptotic impact on the steady-state cost 
(in the cost framework of [33]). The second scaling by y/n in Assumption 12.41 is chosen to make the 
thresholds asymptotically infinite in many-server heavy-traffic diffusion scaling, so that asymptotically 
the thresholds will not be exceeded under normal loading. It is significant that the scaling shows that we 
should be able to simultaneously satisfy both conflicting objectives in large systems. 

We consider an overload incident in which class 1 is overloaded, and more overloaded than class 2 if 
both are overloaded. Hence we primarily focus on the queue difference processes Z?™ 2 . We redefine the 
queue-difference process by subtracting fc™ 2 from Q™, i.e., 

Dl 2 (t) = Q?(t)-kl 2 -r lt2 Q%(t), t>0. (6) 

(Similarly, we write D 21 (t) = r 2 ,iQ 2 (t) — k 21 — <9i(*)-) We now apply FQR using the process _D™ 2 m 
©: if D" 2 (t) > 0, then every newly available agent (in either pool) takes his new customer from the 
head of the class-1 queue. If D™ 2 (t) < 0, then every newly available agent takes his new customer from 
the head of his own queue. 

Let 

A™ A 
P? = and Pl = lim = — !-, i = 1,2. (7) 

Then p™ is the traffic intensity of class i to pool i, and pi can be thought of as its fluid counterpart. 

Our results depend on the system being overloaded. However, in our case, a system can be overloaded 
even if one of the classes is not overloaded by itself. We define the following quantities: 

Qi = 1 J" — and s i = [ m i . 1 = !> 2 ' ( 8 ) 

where (x) + = max{x, 0}. It is easy to see that qfsf = 0, i = 1, 2. Note that is the steady-state of the 
class-i fluid-limit queue when there is no sharing, i.e., when both classes operate independently. Similarly, 
sf is the steady state of the class-i fluid-limit idleness process. For the derivation of the quantities in (j8]) 
see Theorem 2.3 in [46], especially equation (2.19), and §5.1 in [33]. See also §6 in [55] . 

2.4 Conventions About Notation. We use capital letters to denote random variables and stochas- 
tic processes, and lower-case letters for deterministic counterparts. For a sequence {Y n : n > 1} (of 
stochastic processes or random variables) we denote its fluid-scaled version by Y n = Y n /n. The fluid 
limit of stochastic processes Y n will be denoted by a lower case letter y, and sometimes by Y. Let => 
denote convergence in distribution. Then the fluid limits will be expressed as Y n y as n — > 00. 
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We use the usual R, Z and Z + notation for the real numbers, integers and nonnegative integers, 
respectively. Let Rfe denote all /c-dimensional vectors with components in R. For a subinterval / of 
[0,oo), let T>k(I) = 2?(/,Rfc) be the space of all right-continuous Rfe valued functions on I with limits 
from the left everywhere, endowed with the familiar Skorohod J\ topology. We let dj 1 denote the metric on 
T>k(I). Since we will be considering continuous limits, the topology is equivalent to uniform convergence 
on compact subintervals of /. If / is an arbitrary compact interval, we simply write T>k- Let be the 
subset of continuous functions in T>k- Let e be the identity function in T> = X>i, i.e., e(t) = t, t <E I . The 
function G T> will be denoted simply by 0, when the context is clear, or by Oe. 

We use the familiar big-0 and small-o notations for deterministic functions: For two real functions / 
and g, we write 

f(x) = 0(g(x)) whenever limsup \f{x)/g{x)\ < 00, 

x — >oo 

f(x) = o(g(x)) whenever limsup \ f{x)/g{x)\ = 0. 

The same notation is used for sequences, replacing x with n € Z+. 

For a, b € R, let a A b = min {a, b}, a V b= max {a, b}, (a) + = a V and (a)~ ee V —a. For a function 
x : [0, 00) -» R and < t < 00, let 

||x||t EE SUp \X(S)\. 

0<s<t 

Let Y = {Y(t) : t > 0} be a stochastic process, and let / : [0, 00) — > [0, 00) be a deterministic function. 
We say that Y is Op(f(t)), and write Y = Op(f), if ||Y|| t //(i) is stochastically bounded, i.e., if 

lim limsup P (|| Y|| t //(f) > a) = 0. 

We say that Y is op(f(t)) if ||Y||r.//(i) converges in probability (and thus, in distribution) to 0, i.e., if 
||y|| t //(£) => as t — > 00. If /(i) ee 1, then Y = Op(l) if it is stochastically bounded, and Y = op(l) 
if ||Y||t =>■ 0. We define Op(/(n)) and op(f(n)) in a similar way, but with the domain of / being Z+, 
i.e., / : Z + — > [0,oo). These properties extend to sequences of random variables and processes indexed 
by n if the property holds uniformly in n. 

3. Representation of the Fluid Limit. In this section we represent the fluid limit as a solution 
to an ODE which is driven by a FSTP. In contrast to the six-dimensional scaled process in the 
ODE is only three-dimensional. Hence, we start by briefly discussing the dimension reduction in i)3.1l 
Afterwards, we define the FTSP in g3T2] and then present the ODE in 021 We have studied the FTSP 
and the ODE in [3S], to which we refer for more details. In £)3.4I we state three main assumptions. 

3.1 Dimension Reduction. We will making assumptions implying that we consider the system 
during an overload incident in which class 1 is overloaded, and more so than class 2 if it is also overloaded. 
We will thus be considering sharing in which only pool 2 may help class 1. We will thus have both service 
pools fully occupied, with service pool 1 serving only class 1. We will thus asymptotically have Z™ 1 = m™, 
Z21 = and Z™ 2 + ^22 = TO 2- Making a precise statement of this dimension reduction and justifying 
it will be the topic of Thus, in addition to the process X£ in ^ for each n, we also consider the 
essentially three-dimensional six-dimensional processes 

Xq'* ee {Qi, Q2, m'i, Z™ 2 , 0e, — -^"2) in V 6 (9) 

and the associated three-dimensional processes 

A7ee(Qi , ,Q 2 \Zi> 2 ) in 2? 3) (10) 

obtained by truncating the process X 6 ' , keeping only the non-degenerate first, second and fourth coor- 
dinates. We obtain a further alternative representation for the associated three-dimensional fluid-scaled 
processes X^, denoted by X n in $6] see (|4Tj) . 

3.2 The FTSP. Since we consider the system during an overload incident in which class 1 is over- 
loaded, and more so than class 2 if it is also overloaded, we will primarily consider only the one queue 
difference processes Z?™ 2 in ©. The FTSP can perhaps be best understood as being the limit of a family 
of time-expanded queue- difference processes, defined for each n > 1 by 

D n e {T n ,s) = Dl 2 (t Q + S /n), S >0. (11) 
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where X n is the three-dimensional process in (fTU)) and we condition on X n (to) = T n for some deterministic 
vector r™ assuming possible values of X n (to) = (Qi (to), Q 2 (*o)j ^12(^0))- (The time to is an arbitrary 
initial time.) We choose T n so that T n /n — > 7 as n — > 00, where 7 = (qi,q 2 , zi, 2 ) is an appropriate fixed 
state (in three dimensions, because we will have sharing in only one direction. The formal statement 
of the limit for D™ in (fTTj) is Theorem 14.31 Since we divide s in (fTTI) by n, we are effectively dividing 
the rates by n. (See (|4"5|) - (|4"4"1) for the transition rates of Z3" 2 itself.) We are applying a "microscope" 
to "expand time" and look at the behavior after the initial time more closely. That is in contrast to 
the usual time contraction with conventional heavy-traffic limits. See |43j for a previous limit using time 
expansion. 

Let r = 7*1,2 and let 7 = (91, <jfa> #1,2) be a possible state in the three-dimensional state space § = 
[0,oo) 2 x [0, ma]- Directly, we let the FTSP {D(j, s) : s > 0} be a pure-jump Markov process with 
transition rates A_ (7), A_ (7), fj,_ (7) and /U_ (7) for transitions of +r, +1, — r and —1, respectively, 

when -D(7, s) < 0. Similarly, let the transition rates be \^\~f), A+ (7), ^^(7) and ^+^(7) for transitions 
of +r, +1, — r and — 1, respectively, when D(^/,s) > 0. 

We define the transition rates for .0(7) as follows: First, for D(j, s) € (— 00, 0] with 7 = (<7i, 92, £1,2), 
the upward rates are 

A_(7) = Ai, and A^(7) = A*i,2^i,2 + M2,2(m 2 - z\$) + 6 2 q 2 . (12) 
Similarly, the downward rates are 

l^L* (7) = /ii^mx + 6iqi and /i ( T ) (7) = A 2 (13) 

Next, for D(7,s) G (0, 00), we have upward rates 

A^ 1) (7) = A 1 and A^ r) (7) = d 2 q 2 . (14) 

The downward rates are 

M+ (7) = + Mi, 2^1, 2 + ^2,2(^2 - £1,2) + Qiqi and /U+ (7) = A 2- (I 5 ) 

As in §7.1 of [35], we identify important subsets of the state space § = [0, oo) 2 x [0, 777.2]: 

§ b EE { qi - rq 2 = 0}, §+ EE {(71 - rq 2 > 0}, S" = {(?i - r(/ 2 < 0}. (16) 

Let .0(7,00) be a random variable with the steady-state limiting distribution (as s — > 00) of the FTSP 
and let 711,2(7) = P{^{l> 00 ) ^ 0)- It turns out that .0(7, 00) and 771,2(7) are well defined throughout S. 
In § 6 the function 771,2 can assume its full range of values, < 711,2(7) ^ 1> the boundary subset S b is 
where the AP is taking place. For all 7 £ 8 + , 771,2(7) = 1; f°r all 7 £ S", 771,2(7) = 0- I 11 order for §~ to 
be a proper subspace of §, both service pools must be constantly full. Thus, if 7 € § _ , then z\\ = 7771 
and 21.2 + 22,2 = m 2, but <7i and q 2 are allowed to be equal to zero. 

An important role will be played by the subset A of § h such that the FTSP is positive recurrent. The 
AP takes place only in the set A. In Theorem 6.1 of [35] we showed that positive recurrence of the FTSP, 
and thus the set A, depends only on the constant drift rates in the two regions: 

M7) - r(A«( 7 )-M ( T ) (7)) + (Ai 1) (7)-^ 1) (7)) 

*_( 7 ) ee r(A ( !- ) ( 7 )-/ J ( : ) (7)) + (A (1) (7)-M (1) (7)). (17) 
The FTSP {D(-f, s) : s > 0} is positive recurrent if (and only if) the state 7 belongs to the set 

A = { 7 eS:5_( 7 ) >0>M 7 )}. (18) 

Let the other two subsets of § h be 

A+ ee {x e S b I 5+{x) > 0} and A~ = {x € § b | S-(x) < 0}. (19) 

From Theorem 6.2 of [35], we obtain the following lemma, giving the limiting behavior of the FTSP 
for any state in S. 
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Lemma 3.1 (limiting behavior of the FTSP) For all 7 € S and igK, 

Um^ooPiDij, s)<x)^ F(rf, x) = P( 7 , (-cx>, a;]), (20) 

where F(j, •) is a cdf associated with a possibly defective probability measure P(7, •) depending on the 
state 7. Moreover, 

for all 7 G A and a; G M, < F(7, x) < 1 and < 711.2(7) < 1; 

for all 7GS+UA+ and x G M, F(7,x)=0 and 711,3(7) = 1; (21) 

/or a/Z 7 G §~ U A - and x G R, F(7) x) = 1 and 711,2(7) = 0- 

Later in i)9.21 we obtain a proper limiting steady-state distribution for the FTSP for all 7 in § by appending 
states +00 and —00 to the state space R of the FTSP {-D(7, s) : s > 0}. Lemma \3.1\ then implies that 
P(7,R) = 1 for 7 G A, P(7, {+00}) = 1 for 7 G §+ U A+ and P(7, {-00}) = 1 for 7 G S~ U A". 

3.3 The ODE. We can now present the three-dimensional ODE in terms of the FTSP D. Let 

x(t) = (qi(t), q2(t), z\^{t) ) be the solution to the ODE at time t; let x = (gi, g2, £1,2), where ±{t) is the 
derivative evaluated at time t and 

qi(t) = Ai - mi/i^i - 7Ti )2 (x(t)) [^l,2(*)Ml,2 + 2^2,2 (*)M2,a] - 0igi(O 

g 2 (0 = A 2 - (1 - 7ri, 2 (x(i))) [Z2,a(t)/i2,2 + 21,2(^^,2] - 0292(0 (22) 

Zl,2(i) = 7ri,2(x(t))z2,2(*)A*2,2 - (1 - ^^(x^)))^^^)^^, 

with 7Ti.2(x(f)) = P(D(x(t), oo) > 0) for each f > 0, where D(x(t), oo) has the limiting steady-state 
distribution as s — > oo of the FTSP D(7, s) for 7 = x(rj). Theorem 5.2 of [35] shows that the ODE has 
a unique solution as a continuous function mapping [0,oo) into S for any initial value in S. Lemma 13.11 
shows that 7Ti,2(x(0) is well defined for any x(t) in S. 

Equivalently, we have the following integral representation of the ODE in (|2"21 : 

21,2(0 = ^1,2(0) + fi 2 ,2 / 7ri ]2 (x(s))(m 2 - 2i j2 (s)) ds - /ii, 2 / (1 - Tr ly2 (x(s)))z 1 ^{s) ds, 



(i 



gi(0) + Ait - mit - pLi.2 I 
Jo 



qi(t) = qi(0) + Ait- mit- fix t2 / 7ri j2 (x(s))zi > 2(s)) ds 



-^2,2/ 7ri, 2 (x(s))(m2 - 2i, 2 (s))ds- 0i / qi(s)ds, (23) 
Jo Jo 

52(0 = ?2(0) + A 2 t - ^2,2 / (1 - 7ri, 2 (x(s)))(m 2 - zi j2 (s))ds 

Jo 

-Mi,2 / (l-7ri i2 (x(s)))^i,2(s)ds-02 / q2{s))ds. 
Jo Jo 

We will see that the integral representation in (|23l) is closely related to an associated integral represen- 
tation of X n = (Q™ , Q 2 , Zi 2); see (|4"T1) ; X ra is replaced by the deterministic state x and the indicators 
1{£>» 2 ( S )>0} are replaced by 7ii, 2 (x(s)). 

Since the right side of the ODE has 7ii,2(x(0)> the evolution of the ODE beyond t depends on the 
FTSP, specifically, upon the steady-state distribution of that FTSP. The relevant FTSP at time t depends 
on the solution of the ODE at time t, x(t). Given x(t) for some t > 0, we can determine the FTSP 
{D(x(0,s) : s > 0}. Given that FTSP, we can determine the steady-state quantity ttl^x^))- Then 
Ti,2(x(0) appears on the right side of the ODE in (|22l) . determining the future of the ODE. Thus, a key 
result is Theorem 7.1 of [35 , showing that 711,2 (x) is locally Lipschitz continuous. That is used to prove 
that there exists a unique solution to the ODE. To obtain that result, we exploit the representation of 
the FTSP, first as a continuous time Markov chain (CTMC, with discrete state space), and second as a 
quasi-birth- and- death process (QBD), which we obtain for the pure-jump Markov FTSP D by assuming 
that the parameter r = r^a is rational. In [35] we also provided an efficient algorithm to solve this ODE 
coupled with the FTSP, which also exploits the QBD structure. 



3.4 Three Main Assumptions. We now introduce three main assumptions: Assumptions I3.lll3.3l 
below. All three assumptions are assumed to hold throughout the paper. These assumptions are in addition 
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to the four assumptions made in §SJ Assumptions 12. 1112.41 (Here we do not require (UJ.) Our first new 
assumption is on the asymptotic behavior of the model parameters; it specifies the essential form of the 
overload. For the statement, recall the definitions in (J3J, (J5J) and ©, which describe the asymptotic 
behavior of the parameters. 

Assumption 3.1 (system overload, with class 1 more overloaded) 

The rates in the overload are such that the limiting rates satisfy 

(1) 0igf >/ii,a*S. 

(2) qf >r lt2 &. 

Condition (1) in Assumption 13.11 ensures that class 1 is asymptotically overloaded, even after receiving 
help from pool 2. To see why, first observe that, since s 2 > 0, qf > 0, so that Ai > /Lii,imi and p\ > 1. 
Hence, class 1 is overloaded. Next observe that /xi,2S 2 = Mi, 2(1 — P2) 4 ", and that (1 — p 2 ) + is the amount 
of (steady-state fluid) extra service capacity in pool 2, if it were to serve only class-2 customers. Thus, 
Condition (1) in Assumption [3J] implies that enough class- 1 customers are routed to pool 2 to ensure that 
pool 2 is overloaded when sharing is taking place. This conclusion will be demonstrated in £[6] Condition 
(1) in Assumption 13 . 1 1 is slightly stronger than Condition (7) of Assumption A in [33] ■ because here there 
is a strong inequality instead of a weak inequality. 

Condition (2) in Assumption 13.11 ensures that class 1 is more overloaded than class 2 if class 2 is also 
overloaded. This condition helps ensure that there is no incentive for pool 1 to help pool 2, so that Z 21 
should remain at 0. 

Our second assumption is about the initial conditions. For the initial conditions, we assume that the 
overload, whose asymptotic character is specified by Assumption 13.11 is ongoing or is about to begin. 
In addition, sharing with pool 2 allowed to help class 1 has been activated by having the threshold fc™ 2 
exceeded by the queue difference process D™ 2 an d is in process. Thus actual sharing is being controlled 
by the difference process -D™ 2 in ©■ Here is our specific assumption. 

Assumption 3.2 (initial conditions) For each n > 1, P(Z^ 1 (Q) = 0, Q™(0) > a n ,i = 1,2) = 1, 
I"(0)^j;(0)eAUA + U§ + as n -> 00, and D™ 2 (0) L if x(0) G A U A + , 

where {a n : n > 1} is a sequence of real numbers satisfying a n /c n —¥ a, < a < 00, for c n in ([5j; D™ 2 is 
defined in ([6]); X" = (Q™, Q 2 , ^"2)) x (®) * s a deterministic element 0/R3 ; A, A + and § + are the subsets 
o/S in (|18p . (112]) and HH); and L is a proper random variable. 

Since we are interested in times when sharing occurs with pool 2 helping class 1, in Assumption 13.21 we 
assume that the scale of Qi(0) is at least as large as that of the threshold k™ 2 ( so either the threshold 
has already been crossed, or it is about to be crossed). Note that we also assume that D" 2 (0) =>• L, so 
that it is natural to assume that Q 2 (0) has the same order as Q™(0) w.p.l. In Assumption 13.21 we do 
not allow a;(0) in §~, because such an initial condition may activate sharing in the wrong direction, with 
pool 1 helping class 2, causing the system to leave the state space S; see Remark |4. 11 

As noted in §3.3[ in [35j we required that the queue ratio parameter be rational in order to establish 
results about the FTSP and the ODE. 

Assumption 3.3 (rational queue ratio parameter) The queue ratio parameters ?t i2 and ra 1 are rational 
positive numbers. 

Given Assumption 13.31 without loss of generality, we let the thresholds be rational (of the form fc™ 2 = 
m n /k where 7*1,2 = j/k). We conjecture that Assumption 13.31 can be removed, but that condition has 
been used in [3S] to make the pure-jump Markov FTSP a QBD, which in turn was used to establish 
critical properties of the FTSP and the ODE. We use some of these properties in this paper as well. By 
Assumption 13 . 3[ 7*1,2 = j/k for positive integers j and k. The computational efficiency of the algorithm to 
solve the ODE developed in §11 of [35] actually depends on j and k not being too large as well, because 
the QBD matrices are 2m x 2m, where m = max{j, k}, see §6.2 of [3S], and the steady-state of that 
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QBD must be calculated at each discretization step in solving the ODE. Fortunately, simulations show 
that the system performance is not very sensitive to small changes in ri j2 , so that having to be 5 or 10 
seems adequate for practical purposes. 

Relaxing Assumption 13.31 will have practical value only if an efficient algorithm for solving the ODE is 
developed. We remark that computing the stationary distribution of a pure-jump Markov process can in 
general be hard and time consuming, and that we need to compute the stationary distribution of a large 
number of such processes in order to solve the ODE. Hence, the ability to analyze the FTSP as a QBD 
has an important advantage, even if Assumption 13.31 is relaxed. 

4. Main Results. In this section we state the main results of the paper. In £14.11 we state the main 
theorem, establishing the FWLLN via the AP, proving that the (unique) solution to the ODE (|2"2"|) is 
indeed the fluid limit of Xq. In ij4.2l we establish convergence of the stationary distributions, showing 
that the order of the two limits n — > oo and t — > oo can be interchanged in great generality. In £14.31 wc 
establish asymptotic results about the queue-difference stochastic process. We conclude in £14.41 by giving 
a brief overview of the following proofs. 

4.1 The Fluid Limit. We are now ready to state our main result in this paper, which is a FWLLN 
for scaled versions of the vector stochastic process (Xq ,Y s n ), where Xq = (Q™ : Zfj) £ Dq as in ([2]) and 
Yg" = (A™, S™j,U™) £ T>s, i,j = 1, 2, where A™(t) counts the number of class-i customer arrivals, S™j(t) 
counts the number of service completions of class- i customers by agents in pool j, and U^(t) counts the 
number of class-z customers to abandon from queue, all in model n during the time interval [0,t]. For 
the FWLLN, we focus on the scaled vector process 

ft-.^E^ft-.y;), (24) 

as in (HJ). Recall that Assumptions I3.HI3.3I are in force, and assumed to be in force throughout the paper. 



Theorem 4.1 (FWLLN) As n ^ oo, 

(X£,Y s n )^ (x 6 ,y s ) m 2? 14 ([0,oo)), (25) 

where (xe,ys) is a deterministic element of Cu([0, oo)), xq = (qi,Zij) and ys = (<2j, Sij, it,-), i = 1, 2; 
j = 1, 2; Z2.1 = S2.1 = mi — zi t i = TO2 — 2:2,2 — Zi,2 = 0e; x = (91,^2,^1,2) is the unique solution to 
the three-dimensional ODE in (|22|) mapping [0,oo) into §. The remaining limit function ys is defined in 
terms of xq : 

o,i{i) = Xit, s it j(t) = Zij(s)ds,Ui(t) = 9i qi(s)ds for t > 0, i = l,2; j = l,2. (26) 

Jo Jo 



We prove Theorem l4.1l bv showing in £|5.2l that the sequence {(Xq, Yg 1 ) : n > 1} is C-tight in 2?i4([0, 00)) 
and by showing subsequently that the limit of every convergent subsequence must take values in § and 
be a solution to the ODE (|2"2"]h which has a unique solution by Theorem 5.2 of [55] , 



4.2 WLLN for the Stationary Distributions. Under the FQR-T control operating during a sin- 
gle overload incident of unlimited duration, the six-dimensional stochastic process Xq = (Q™, Z^pi, j = 
1,2) is a a positive recurrent irreducible CTMC for each n. Hence, X§ = n~ 1 X 6 l has a unique steady- 
state (limiting and stationary) distribution Xq(oo) for each n. In [35] it is shown that, under the 
assumptions here, excluding Assumption ^. 21 (which plays no role), there exists a unique stationary point 
x* = (ql , (72, z\ 2 ) in the state space § to the three-dimensional limiting ODE in (|2"2"|) . where 

» _ 6> 2 (Ai - toi^i ; i) - r-i, 2 fli(A 2 - TO 2 /i2,2) 

f 1,2*7 1/42,2 + P2/tl,2 
Ai - TOi/ti,! - Ml,2-2* 2 , * A 2 - /Z2, 2(^-2 ~ Z* 2 ) 

q x = ■ and q 2 = — . 

Moreover, x* is globally asymptotically stable and x(t) converges to x* exponentially fast as t — > 00. Let 
Xq be the six-dimensional version of x* = (ql , q%, z* 2 ) in (|2T|) . i.e., 

^6 = {Q*)Qu,'muz*2,0,m2 - z{ 2 ) for x* = (q*, q%, z* 2 ). (28) 
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Theorem 4.2 (WLLN for the stationary distributions) Under the assumptions here, excluding Assump- 
^077, 13.21 Xq G S for x% in (|28|) and Xq(oo) Xq inRe as n — > oo. 



We prove Theorem 14.21 in ijlO.ll As a consequence of Theorems 14.11 and 14.21 we obtain the following 
corollary. 



Corollary 4.1 (interchange of limits) Under the assumptions here, including A ssumvtion Yi. 21 for each 
continuous bounded function f : — > R, 

lim lim E[f(X n (t))] = lim lim E[f(X n (t)} = f(x* 6 ), 

n— >oo t— foo t— foo n— >oo 

where Xq is defined in (|28[) . 



Proof. The iterated limit on the left holds by virtue of Theorem l4.21 The iterated limit on the right 
holds because of Theorem 14.11 together with the fact that Xq is a globally asymptotically stable stationary 
point for the fluid limit, by jH and Theorem 8.3 of [35]. □ 



Remark 4.1 (starting in §~) It is significant that the limit interchange in Corollary 14.11 is not valid 
throughout §. If Assumption 13.21 holds, except that x(0) G S _ , then 91 (0) — ri j2 92(0) < 0. Together 
with Assumption 12. 2\ that implies that, in some regions of S~, d.2,1 = r^, 192(0) — 9i(0) > 0; that can 
hold in S~ because r%, 292(0) can be larger than 91 (0). In those cases we have P(Z?" 2 (0) < 0) — > 1 and 
P(D^ 1 (Q) > k% tl ) -> 1 as n -> 00. If we assume that P(Z£ 2 (Q) = Z£ x (0) = 0) = 1 for all n > 1, which is 
consistent with Assumption 13.21 then, asymptotically, we will initially have sharing the wrong way, with 
pool 1 helping class 2. By the continuity, there will be an interval [0, S] for which 

inf {da,i(t)}>0. 

{0<t<<5} 

Hence, asymptotically as n —> 00, there will rapidly be sharing with pool 1 helping class 2. It can be 
shown that there exists 6 > and e > such that P(Z 2 1 ' 1 ((5) > e) — > 1 as n — > 00. This shows that the 
limit interchange is not valid for every initial condition in 



4.3 The Limiting Behavior of the Queue Difference Process. In this section we present 
important supplementary results that help "explain" the AP, which takes place in A. The following 
results are not applied in the proof of Theorem l4.11 but are also not immediate corollaries of the FWLLN; 
their proofs are given in SJTUJ 

For each n > 1, let Tg be a random state of X£ that is independent of subsequent arrival, service and 
abandonment processes. 



Theorem 4.3 If T%/n =*> j 6) where 7 6 = (qi, 92, mi, Zi^, 0, m 2 — 21,2) with 7 = (91, 92, Zi.,2) € A C M 3 
for A in (|18l) and D™(Tg,0) => D(7,0) in R as n — > 00, where D™ is the time-expanded queue-difference 
process in (|11[) and D is the FTSP in Q'3.21 then 

{£>™(r n ,s) : s > 0} {P»(7,s) : s > 0} m P as (29) 

i.e., we /iawe convergence of the sequence of time-inhomogeneous non-Markov processes {D™(Tq) : n > 1} 
to t/ie limiting time-homogeneous pure-jump Markov process D. 



The next results are about the queue-difference process D" 2 itself (as opposed to the expanded queue 
difference process P>"). Recall the definition of stochastic boundedness in ^2.41 Recall also that tightness 
in R is equivalent to stochastic boundedness in R, but not in T>. 

Theorem 4.4 (stochastic boundedness of -D" 2 ) If %(to) G A for some to > 0, then there exists t% > to 
such that x(t) G A for all t G [to, £2] an d for all t\ satisfying to < t\ < t 2 the following hold: 

(i) {Di 2 (t) : n > 1} is stochastically bounded in R for each t satisfying t\ <t<t 2 . 

(ii) {{D'i 2 (t) : t G J} : n > 1} is neither tight nor stochastically bounded in T>(I), I C [£0,^2]- 
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(Hi) For any sequence {c n : n > 1} satisfying c„/logn — > oo as n — > oo, it holds that 

sup {D\\ 2 (t)/c n } ^ as rw oo. (30) 

tl<t<t 2 

If x(t) G A for all t G [iojOo) the above statements hold for any finite t 2 > to. 

As an immediate corollary to (|30|) in Theorem 14.41 we have the following SSC of the queues. In 
particular, that claim implies SSC of the fluid and diffusion scaled queues when the fluid limit x is in A. 

Corollary 4.2 (SSC of queue process in A) For the interval [tijta] in Theorem 14.41 
djx (Qi , fx^Q^/cn =>■ in T>([ti,t2\) as n — > oo, for every sequence {c n : n > 1} satisfying 
c n /logn — y oo as n — > oo. If x(0) G A and we consider the interval [0,tz\, then the result is strengthened 
to hold on [to,t 2 ] = [0,t 2 ]. 

Since the sequence of queue-difference processes is not V tight, by virtue of Theorem 14.41 we cannot 
have convergence of these processes in T>. However, we can obtain a proper limit for the tight sequence 
of random variables {D" 2 (t) : n > 1} in R for each fixed t G [£1,^2] by exploiting the AP. See [33] for a 
similar result. 

Theorem 4.5 (pointwise AP) Consider the interval [tijta] in Theorem \4--4\ Then -D" 2 (t) =^ D(x(t), 00) 
in R as n —¥ 00 for each t, t\ < t < f2, where D(x(t), 00) ftas t/ie limiting steady-state distribution of the 
FTSP D(j,s) forj = x(t). 

Remark 4.2 (tightness after hitting times of A) Observe that all the results above will hold at time 
to = if x(0) G A because of the assumed convergence of D™ 2 (0) in Assumption 13.21 However, even if 
x(0) G A, then the fluid limit might leave A eventually, and later return to A at some time to- Let to > 
be a hitting time of the set A, assuming x(t) is not in A just before to, i.e, x(s) £ A for all s G (to — e, to), 
for some e > 0. Our results hold for all t\ G (£01*2] for the hitting time to and for t 2 as above. 

4.4 Overview of the Proofs. The rest of this paper is devoted to proving the five theorems above: 
Theorems 14. Hl4~5l We prove Theorem l4.1l in §<|5][9l Toward that end, in ^J5]we establish structural results 
for the sequence {(X$, Y 8 n ) : n > 1}, where Xg = (Qf, Z?J) G V 6 as in © and Y a n = (A™, S? d ,U?) G V 8 , 
i,j = 1,2 and the associated fluid-scaled process (Xq,Y^) in (|24| . In i)5.1l we construct the stochastic 
processes (Xg , ig") in terms of rate-1 Poisson processes. In H5.2I we show that the sequence of stochastic 
processes {(Xg, F 8 n ) : n > 1} is C-tight in V14 and, consequently, there are convergent subsequences with 
smooth limits. In ^we show that the representation established in fc|5.1l can be simplified under Assump- 
tions 13. 1113.31 reducing the essential dimension from 6 to 3. The final three-dimensional representation 
X n in gj) there explains the form of the ODE in (|22]) . 

In f}7]we present supporting technical results to be applied in following sections. We start by introducing 
auxiliary frozen queue difference processes in $7.1i We construct useful bounding processes in jj £ 17.21 17.31 
and 17.41 These are primarily for quasi-birth-and-death (QBD) processes, because we exploit a QBD 
representation for the FTSP; see §6 of [35] for background. We establish extreme value limits for QBD 
processes in £17.51 In £[8] we exploit the technical results in £J7| to prove three theorems stated in £j6j 

Given the tightness established in H5.21 we prove the main Theorem 14. II by characterizing the limit of all 
convergent subsequences in £]§! Given the SSC established in §£j5][5]and given that the three-dimensional 
ODE in £13.31 has been shown to have a unique solution in [35 , it suffices to show that the limit of any 
subsequence must almost surely be a solution to the ODE. For that last step, our proof in £j9] follows 
Hunt and Kurtz [17j . which draws heavily upon Kurtz [25 . It exploits our martingale representation in 
theorem 16.31 and basic properties of random measures from [25j . We also have developed an alternative 
proof exploiting stochastic bounds. It is given in the appendix. Finally, in flOl we prove Theorems 14.21 
1431 

5. Preliminary Results for X£. In this section we establish preliminary structural results for the 
vector stochastic process (Xg, F 8 "), where Xg = (Q™, Z^) G V 6 as in © and F™ = (A\ l , S^U?) G V s , 
i,j = 1,2 and the associated fluid-scaled process (Xg ,Y%) in (|24| . The results in this section do not 
depend on Assumvtions \3.1 iTOl We do impose the many-server heavy-traffic scaling in £|2.3I 



: An Averaging Principle 

Mathematics of Operations Research xx(x), pp. xxx-xxx, (c)200x INFORMS 



13 



In i j5.ll we construct the stochastic processes (Xq,Y%) in terms of rate-1 Poisson processes. In Sj5 
we show that the sequence of stochastic processes {(Xq, ig ) : n > 1} is C-tight in P14. In Corollarv l5.ll 
we apply the tightness to deduce smoothness properties for the limits of convergent subsequences. 

5.1 Representation of X£. In this section we develop representations for the basic CTMC X£ 
with the FQR-T control. At first in this section we do not require Assumptions I3.H3.31 so that we can 
have sharing in either direction, but in only one direction at any time. Let (A™, A 2 ) be the pair of fixed 
positive arrival rates in model n, which here are unconstrained. 

Following common practice, as reviewed in §2 of |32j . we represent the counting processes in terms 
of mutually independent rate-1 Poisson processes. We represent the counting processes A™, Sfj and U- 1 
introduced in the beginning of £)4.1I as 

A?(t) = Nf(X"t), S&(t) = Nfj J Z^(s) ds^j , U?(t) = N? U J Qf(s) ds\ , (31) 

for t > 0, where iVf, N?j and AT" for i = 1,2; j — 1,2 are eight mutually independent rate-1 Poisson 
processes. 

We can then obtain a general representation of the CTMC Xg, which is actually valid for general 
arrival processes with arrivals one at a time. Let 5™ = S 1 ™ x + S% 1 , S 2 = $12 + ^2 2 an d S n = S" + S%- 
Paralleling ©, let D^t) = r 2 ,iQ^(t) - fc£ x - QJ(t). 

Theorem 5.1 (general representation of X§) Here Assumvtions \3. 1113.31 are not in force. For each n > 1, 
the stochastic process Xg is well defined as a random element of 2?g by (|3 1[) and 

Ql(t) = Ql (°) + / 1 {Z 1 " 1 (s-) + Z 2 " 1 (s-)=m7} dA" (s) - / l{D? 2 (s-)>0,Z2 1 (s-)=0,Q?( S -)>0}dS n (s) 

Jo Jo 

rt 

1 {{Qi(s-)>o}n({zi 1 (s-)>o,Di 1 (s~)<o}u{Z2 1 ( s ~)=o,D? 2 (s-)<o})} dSi(s) - U?(t), 



ZUV = Z U°)+ hz ?1 (s-) + z 5l{ s- )<m ' n dAUs) 
Jo 

1 {({z^ 2 ( s -)>o}u{Di 1 (s-)<o})n{Q'i( s -)=o}}dSl 1 (s) - j l {jD » i(s _ )>0 ,z» 2 ( s _) = o}(i5r,i(s), 

= ^".2(0)+ / 1 {D^ 2 (s-)>0.Z 2 " 1 (s-)=0,Qf (»-)>0} dS% 2 (s) 

Jo 

1 {{sr,2(«-)<o}u{zj_ 1 ( s -)>o}}rf5 , r,2( s )- 



Symmetry yields the parallel definitions of Q^it), Z^itf) an d ^2i{t) from Qi{t), Z™ ^t) and Zi 2 {t) by 
simply switching the subscripts 1 and 2. 

Proof. Just as in Lemma 2.1 of [32] , we can justify the construction by conditioning on the initial 
values (the first term in each display) and the counting processes. With these sample paths specified, 
we recursively construct the sample path of Xq. By applying mathematical induction over successive 
transition epochs of Xq , we show that the sample paths are right-continuous piecewise-constant functions 
satisfying the equations given. 

To explain Q", the second term represents the increase by 1 at each class- 1 arrival epoch when service 
pool 1 is fully occupied; otherwise the arrival would go directly into service pool 1. The third term 
represents the decrease by 1 when any server completes service and sharing with pool 2 helping class 1 
active; that requires that the class-1 queue length be positive (Q™ (s) > 0); sharing with pool 2 helping 
class 1 occurs when both £)™ 2 ( s ) > and Z% i(s) = 0. The fourth term represents the decrease by 1 
when any pool-1 server completes service, provided that again the queue length is positive (Q"(s) > 0). 
There are two scenarios: (i) {Z^s) > 0,D% A (s) < 0} and (ii) {Z'i 2 (s) = 0,D£ 2 (s) < 0}. In the first, 
pool 1 is helping class 2, so type-1 servers take from queue 1 only when D 7 2 l x (s) < 0. The second scenario 
is the relative complement within the event {Z%i(s) — 0, Q\ l (s) > 0} of the event in the third term, i.e., 
pool 2 is allowed to help class 1, but D" 2 (s) < 0, so that only type-1 servers take from queue 1 at time s. 
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To explain Z^-y, the second term represents the increase by 1 which occurs at each class- 1 arrival 
epoch at which service pool 1 has spare capacity (Z" x (s) + Z 21 (s) < to"). The third term represents 
the decrease by 1 that occurs when a server in pool 1 completes service of a class-1 customer, with pool 
1 not helping class 2 (({Z{\ 2 (s) > 0} U {D 21 (s) < °})) when the class-1 queue is empty (Q™ (s) = 0). 
The fourth term represents the decrease by 1 that occurs when a server in pool 1 completes service of a 
class-1 customer, when pool 1 is helping class 2 ({D 2 1 (s) > 0, Z™ 2 (s) = 0}). 

To explain Z" 2 , the second term represents the decrease by 1 that occurs when a server in pool 2 
completes service of a class-2 customer, when class 2 is helping class 1 ({D" 2 (s) > 0, ZQ x (s) = 0, Qi(s) > 
0}). The third term represents the decrease by 1 that occurs when a server in pool 2 completes service 
of a class-1 customer, when class 2 is not helping class 1 ({D™ 2 (s) < 0} U {Z 21 (s) > 0}). 

Since the model is fully symmetric, the processes Q 2 , Z 22 and Z 21 are the symmetric versions of Q 2 , 
Z 22 and Z 21 , respectively, with the indices 1 and 2 switched. □ 

5.2 Tightness and Smoothness of the Limits. We do part of the proof of Theorem 14 . 1 1 here by 
establishing tightness. For background on tightness, see [5J [321 131] ■ We recall a few key facts: Tightness of 
a sequence of fc-dimensional stochastic processes in T>k is equivalent to tightness of all the one-dimensional 
component stochastic processes in V. For a sequence of random elements of C-tightness implies T>- 
tightness and that the limits of all convergent subsequences must be in see Theorem 15.5 of the first 
1968 edition of [6J. 

For the rest of this section, we have the following restricted version of Assumption 13.21 

Assumption 5.1 {weakening of Assumvtion W^ As n ^ oo, 

X n (0) x(0) as n ^ oo, 
where x(0) is a deterministic element 0/R3. 

Theorem 5.2 The sequence {(X|\Y 8 n ) : n > 1} in ([24j) is C-tight in V u . 

Proof. It suffices to verify conditions (6.3) and (6.4) of Theorem 11.6.3 of [IS]. Hence, it suffices to 
prove stochastic boundedness of the sequence of stochastic processes evaluated at time and appropriately 
control the oscillations, using the modulus of continuity on C. We obtain the stochastic boundedness at 
time immediately from Assumption 15.11 



We now show that we can control the oscillations below. For that purpose, let w(x, T) is the modulus 
of continuity of the function x £ T>, i.e., 

w(x,C,T) = sup{|z(t 2 ) - x(h)\ : < h < t 2 < T, \t 2 - h\ < Q. (32) 
Using the representations in £|5.1[ for t 2 > t\ > we have 

\Q n (t2) - Q n (ti)\ < ^i^)"- 4 "^) + s n (t 2 )-s n (h) + s^jh) ~ s^jt,) + u?(t2)-utth) _ 

' ' — n n n n 

and similarly for Q 2 . Hence, for any £ > and T > 0, 

w{Q n Jn, C, T) < w{Al/n, C, T) + w(S n /n, C, T) + w(S? tl /n, C, T) + w(U?/n, C, T). 

Then observe that we can bound the oscillations of the service processes S™j by the oscillations in the 
scaled Poisson process N?Jn-). In particular, by (I5T1) . 

w(S^/nX,T) < w(Nl 3 (n^m r )/nX,T) < w(N^ j (n-)/n, c£, T) (33) 

for some constant c > 0. Next for the abandonment process U™, we use the elementary bounds 

Q?(t) < Qm + Am, 

\U?{t 2 ) - U?( tl )\ = \Ni{0i r Q^(a) ds\ < |JVi(n6(Q?(0) + A«{T)){t 2 - h))\. 



Let qM = 2(gj(0) + T), where Q"(0) ^> 9i(0) by Assumption 15.11 and let B n be the following subset of 
the underlying probability space: 

B n = {Qi(0) + A"(T) < q bd }- 
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Then P{B n ) 1 as n — > oo and, on the set B n , we have 

w(U?/nX,T) < w(N?(nq bd -)/nX,T) < w(N?(n-)/n,c(,T) (34) 
for some constant c > 0. 

Thus, there exists a constant c > such that, for any r\ > 0, there exists no and ( > such that, for 
all n > no, P(B n ) > 1 — 77/2 and on B n 

2 2 

w(Q?/n, C, T) < w(N?(n-)/n, c£ T) + 2 £ £ (n-)/n, <, T) + w{N?{n-)/n, <£, T). 

However, by the FWLLN for the Poisson processes, we know that we can control all these moduli of 
continuity on the right. Thus we deduce that, for every e > and 77 > 0, there exists £ > and no such 
that 

P(w(Q?/nX,T) > e) < n for all n > n . 
Hence, we have shown that the sequence {Qf} is tight. 

We now turn to the sequence {Z™ 2 }. Let A™ 2 (t) denote the total number of class- 1 arrivals up to time 
t, who will eventually be served by type-2 servers in system n. Let A™ 2 = ^4™ 2/ 71 an( i ^12^) = Si2(t)/ n i 
for S[ l 2 (t) in {31]). Since 

ZUt) = ZUQ) + Al 2 (t) - Sl 2 (t), 

we have 

\zu^) - zUh)\ < K2(h) - Ai 2 ( tl ) + sut 2 ) - SUh). 

However, for A™ in (f3"Tj) . 

Al 2 (t 2 ) - A^ih) < AUt 2 ) - AUti). 

Since ^4" =^> Aie in V, the sequence {^4"} is tight. Together with (|33|) . that implies that the sequence 
{Z™ 2 } is tight as well. Finally, we observe that the tightness of {fg™} follows from ([531 . (fM)) and the 
convergence of A™. □ 

Since the sequence {(Xq,Y% ) : n > 1} in (|2^|) is C-tight by Theorem 15. 2[ every subsequence has a 
further subsequence which converges to a continuous limit. We now apply the modulus-of-continuity 
inequalities established in the proof of Theorem 15.21 to deduce additional smoothness properties of the 
limits of all converging subsequence. 

Corollary 5.1 If (Xq,Ys) is the limit of a subsequence of {(Xg ,Y^) : n > 1} in T>u, then each 
component in T>, say Xi, has bounded modulus of continuity; i.e., for each T > 0, there exists a constant 
c > such that 

w(Xi,C,T)<cC w.p.l (35) 
for all C > 0. Hence (Xq,Ys) is Lipschitz continuous w.p.l, and is thus differ entiable almost everywhere. 



Proof. Apply the bounds on the modulus of continuity involving Poisson processes in the proof of 
Theorem [5JH above. For a Poisson process AT, let N n = ^/n(N n - e), where N n (t) = N(nt)/n, t>0. By 
the triangle inequality, for each n, £, and T, 

w(N n , C, T) < w ^ Nn £ T > + W U C, T) C as n -> 00. 

Since, w(x, £, T) is a continuous function of x for each fixed C and T, we can apply this bound with the 
inequalities in the proof of Theorem 15.21 to deduce (|35|) . □ 



6. Structural Simplification. We now exploit Assumptions 13 . lti3 . 31 to simplify the representation 
established in ij5.ll above, reducing the essential dimension from 6 to 3, following the plan described in 
i j3.ll We first establish this dimension reduction over an interval [0, r] and later, after Theorem 14.11 has 
been proved over the same interval [0, r], we show that all the results here, and thus Theorem 14.11 too, 
can be extended to the interval [0,oo). 

Let 

T n = inf{* > : Z r 2 \{t) > or Q"(<) = or Q 2 l {t) = 0}. (36) 
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By Assumption 13.21 both queues are initially strictly positive (so there is no idleness in either pool) and 
■^2 i(0) = 0- Hence, T n > for each n > 1. Theorem 15.11 implies the following reduction from six 



dimensions to three over [0,7^™]. (Recall the convention about notation in £13.11 ) Let = denote equality 
in distribution for processes. 

Corollary 6.1 Under Assumptions X% = X%<* = (Q™, Q™, m'{ x , Z™ 2 , 0, m' 2 l - Z{\) over 

[0,V], where 



??(*) = O?(0)+A?(t)- / 1 { ^ 2(S _ )>0} ^' 1 ( S )- / l(D r2 H<o)^iW^i"W 

J Jo 

= Q?(0) + N?(X}t) - NU^mV) - Nf t2 L 2 J l { n U s)>o}Z?, 2 (s)) da) (37) 
-^2,2 (m2,2 f l{D 1 - a W>o}("»5 - ^" 2 (a)) da) - AT (V jf da) , 

/5(t) = qj(o) + A n 2 {t) - [ i {Dl - a(i -)<o} d^y*) - / i{ Dl - a (.-)<o} ds? j3 00 - U2V) 

Jo Jo 
± QJ(0) + JV|(AJt) - iV| i2 L 2 , 2 jf* l {Dr 2(s) <o}(m 2 l - Z£ 2 (a)) da 

-^1,2 (mi,2 y l{o ? , 2 W<o}^r, 2 (s) da) - (#2 jf* 05(a) « 



(38) 



^ l2 (i) = ZIM+ hD? 2 (s-)>o}dSl 2 (s)- l {D ^ (s _ )m dS^ 2 (s) 
Jo Jo 

= Z[\ 2 (0) + N§ >2 ^/i 2 , 2 J l {DUs>0} (m% - Zl 2 )(s) to 
-K2 (viaf hn l2 ( s) <o}Z? }2 (s)ds\ . 



(39) 



Note that Z™ 2 (t) might be equal to zero for some or all t in [0,7^™]. We next prove that T n is bounded 
away from asymptotically, i.e., that there exists a t > such that P(Tq > r) — > 1 as n — > 00. We do 
so in two parts (both proved in jJSJ): 

Theorem 6.1 (no sharing in the opposite direction) There exists t > such that \\Z 21 \\ T as n — > 00. 
Theorem 6.2 (positive queue lengths) For t in Theorem ] 6. 11 



P l^inf minjQ^i),^^)} > OJ ->• 1 as n -J- 00. 

As an immediate consequence of Theorems 16.11 and 16.21 above, we obtain the following SSC result. 

Corollary 6.2 (SSC of the service process) For t > in Theorem 16.11 P(Tq 1 > t) — > 1 as n — > 00, 
where T n is defined in i.e., 

(m? - Z^, Z 2 " l5 m™ - Z£ a - Z 2 n 2 ) =» (0, 0, 0) 2? 3 ([0, r]) a SI ^ M . 

We make two important remarks about Corollary 16.21 First, the limit holds without any scaling. 
Second, here we do not yet show that a limit of Z™ 2 as n — > 00 exists. We only show that, when 
analyzing the four service processes Z™j, it is sufficient to consider Z™ 2 . 

Recall that dj 1 denotes the standard Skorohod J\ metric and Xq '* is the essentially three-dimensional 
process defined in ©. The following corollary is immediate from Corollary [ 



Corollary 6.3 (Representation via SSC) As n -> 00, d Jl (A 6 n , Xg '*) =► in X> 6 ([0, r]), for Xg'* in ©, 
t in Theorem^ and (Q?, Q%, Z^ 2 ) in §J}- 
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We now obtain further simplification using a familiar martingale representation, again see [35]. Con- 
sider the representation of X£ in (l3"7]) - (l39l) above, and let 

M^ a (t) ee JV?(A?t) - \?t, 

M^ u {t) ee N? U [ QUs)) ds) - 9 t [ QUs) ds, (40) 



M^ s (i)EE/Y^(J- 2 (i))-J^(t), 
where Jl l 2 (t) are the compensators of the Poisson-processes iV/ 2 (i) m (|3"7) - (l3l?l) , i = 1,2, e.g., 



^r.2W = /"i,2 / i { ^ 2 ( S )<o}^r, 2 (s)rfs. 

JO 

The quantities in (|40p can be shown to be martingales (with respect to an appropriate filtration) ; See [33] . 
The following lemma follows easily from the FSLLN for Poisson processes and the C-tightness established 
in Theorem [ 



Lemma 6.1 (fluid limit for the martingale terms) As n — > oo, 
n 



- 1 (Mf'%M 2 n '°,M 1 n ' u ,M 2 ri '",M^ s ,M 2 n ^) ^ (0e,0e,0e,0e,0e,0e) m2? 6 ([0,r]). 



Proof. By Theorem 15.21 the sequence {Xg : n > 1} is tight in T>. Thus any subsequence has a 
convergent subsequence. By the proof of Theorem 15.21 the sequences {J^j/n} are also C-tight, so that 
{Ji'j/n}, i = 1,2, all converge along a converging subsequence as well. Consider a converging subsequence 
{X n } and its limit X, which is continuous by Theorem l5.2l Then the claim of the lemma follows for the 
converging subsequence from the FSLLN for Poisson processes and the continuity of the composition map 
at continuous limits, e.g., Theorem 13.2.1 in [45]. In this case, the limit of each fluid-scaled martingale 
is the zero function Oe € V, regardless of the converging subsequence we consider, and is thus unique. 
Hence we have completed the proof. □ 

Hence, instead of X^ (the relevant components of X$'*) in ([3T|) - ([3^| . we can work with X n = 
(QJ,QS,Z? a )for 



^r, 2 W = ^r,2(0)+^2 / l {D « 2(s)>0} (m2-Z? >2 (s))ds-» h2 / l {D n 2(s)m Z^ 2 (s)ds, 
Jo Jo 

QJ(t) EE QJ(0) + AJf - fr$t - Ml,2 / l{D~ 2 (s)>0}Zl 2 {s)) ds 

Jo 

-^2,2 f l { zjf aW >o}(^ -Zl^ds-Ox [ Q?(s)ds, (41) 
Jo ' Jo 

Q n 2 (t) ee Q 2 l (0) + \ n 2 t - ^,2 / l {D ~ 2[s) < 0} {rh n 2 - Z[\ 2 (s)) ds 

Jo 

- fJ>l,2 l{L>^ 2 ( s )<0}^r 2 ( s ) ds - °2 / Q 2 (s))ds. 



Theorem 6.3 As n -> oo 7 dj^X^X") in X> 3 ([0,t]) as n oo, where X£ is defined in (j57 fi -([55 ]) . 
X" is defined in (|41l) and r is as m Theorem \6A\ 



PROOF. It suffices to show that M n =>• in X>3([0, r]) as n -> oo, where 

M n ee X£ - I n . (42) 



However, M n by virtue of Lemma 16.11 above by the continuous mapping theorem with addition at 
continuous limits. □ 

As a consequence of Theorem 16.31 henceforth we can focus on X n in (|4"Tj) instead of X£ in ([3T| - ([591) . 
After we prove Theorem 14. 11 we can extend the interval [0,r] over which all the previous results in this 
section hold to the interval [0,oo). 

Theorem 6.4 (global SSC) All of the results above in this section extend from the interval [0,r] to the 
interval [0, oo). 
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We prove Theorems 16. II and 16. 21 in after establishing supporting technical results in In $8.31 we 
prove Theorem 16. 41 under the assumption that Theorem 14.11 has been proved over [0,r], which will be 
done in by showing that the interval over which the conclusion is valid can be extended from [0, r] to 
[0, oo), once Theorem 14. ll has been proved over [0,t]. That will imply that Theorem 14.11 then holds over 
[0, oo) as well. 

7. Supporting Technical Results. In this section we establish supporting technical results that 
we will apply in future sections. In H7.ll we introduce a frozen queue difference process, which 'freezes" 
the state of the slow process (X n in (|41|l ). so that the fast process (the queue difference process £>™ 2 
in (O) can be considered separately. Like the FTSP in $3.21 the frozen process is a pure-jump Markov 
process. To carry out the proofs, we exploit stochastic bounds. Hence, we discuss them next in §£ 17.21 17.31 
and 17.41 Since the FTSP and the frozen processes can be represented as QBD processes, as indicated in 
§6 of 35], we next establish extreme value limits for QBD's in £ 17.51 We establish continuity results for 
QBD's, used in the alternative proof of characterization, in E1A.31 

7.1 Auxiliary Frozen Processes. Our proof of Theorem [HT] will exploit stochastic bounds for the 
queue difference process D 2 i . Our alternative proof of Theorem 14.11 in the appendix we will exploit 
similar stochastic bounds for the queue difference process D™ 2 . These processes D 2 i and £>™ 2 are non- 
Markov processes whose rates at each time t are determined by the state of the system at time t, i.e., 
by JTg l (i), and is thus hard to analyze directly. In order to circumvent this difficulty, we consider related 
auxiliary processes, with constant rates determined by a fixed initial state T n = Xg(0). The construction 
is essentially the same for the two processes D 2 i and D™ 2 . Since we are primarily concerned with -D™ 2 , 
we carry out the following in that context, with the understanding that there is a parallel construction 
for the process D 2 l ■ 

When we work with £>™ 2 , we exploit the reduced representation involving X n in (|4"Tj) . Let D 1 j(T n ) = 
{D n f {Y n ,t) : t > 0} denote this new process with fixed state T n = (Q? , Q% , Z? 2 ) . Conditional on T n , 
D r j{T n ) is a QBD with the same fundamental structure as the FTSP defined in £13. 21 We use the subscript 
/ because we refer to this constant-rate pure-jump Markov process as the frozen queue- difference process, 
or alternatively, as the frozen process, thinking of the constant transition rates being achieved because 
the state has been frozen at the state T™. 

As in i j3.21 the rates of the frozen process are determined by T n = (Q™, Q 2 , Z™ 2 ) an d by its position. 
Also, as in H3.21 let r = n,2- The frozen process D^(T n ) has jumps of size +r, +1, —r, —1 with respective 

rates A^^T"), \+\n, T n ), ^ W (n,r™), (^(n,!™) when DJ < 0, and jumps of size +r,+l, -r,-l with 

respective rates \ { +\n,T n ), \ ( +\n,T n ), ^ {n, T n ), ^\n,T n ) when D n f > 0. 

Analogously to p^|) - p3|) . in the non-positive state space these rates are equal to 

A a) (n,r") = A™ and A^T") = Mi,2^i" )2 + /fc, 2 (™2 - Z£ 2 ) + 8 2 Q r 2 \ 
^ (n,T n ) = Mi,irni + 0iQi and ^\n, T n ) = A 2 \ 
and in the positive state space, these rates are equal to 
A^(n,r") = A? and X^\n,T n ) = 9 2 Q 2 , 

H { l\n,T n ) = MM m? +Mi,2^ 2 + ^A™2 - Z[\ 2 ) + BiQl and »%\n,T n ) = A J. 



(43) 



(44) 



Using these transition rates, we can define the drift rates for D n (T n ), paralleling (fl~7|) . Let these drift 
rates in the regions (0, oo) and (— oo, 0] be denoted by S"(T n ) and <5™(r™), respectively, Then 

Sl(T n ) = [A™ - + (M2,2 - Vi,2)Z? i2 (t) - f i 2 ,2^(t) ~ OiQW)] - '[AS - 2 Q^(t)], 

5^(T n ) ee [A™ - Mi.im? - 9 1 QUt)} - r[A™ + ( M2)2 - m,u)Z? t2 (t) - ^ 2 m n 2 - 6 2 Q^(t)}. ' 4 " ' 
Just as for the FTSP, DJ(T n ) is, conditional on T n , a positive recurrent QBD if an only if 

tf"(r«) < < <5!!(r n ). (46) 

We will frequently apply this constant-rate pure-jump Markov process with T n being a state of some 
process, such as X n {t). We then write D n f {X n [t)) = {D'J(X n (t), s) : s > 0}, where it is understood that 

D n f {X n {t)) = D](T n ) under the condition that T n = X n (t). 
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It is important that this frozen difference process DJ(T n ) can be directly identified with a version of 
the FTSP defined in ^3.21 because both are pure-jump Markov processes with the same structure. Indeed, 
the frozen-difference process can be defined as a version of the FTSP with special state and basic model 
parameters A™ and m™, and transformed time. In order to express the relationship, we first introduce 
appropriate notation. Let D(Xi, m 3 7, s) denote the FTSP defined as in H3.2I with transition rates in 
(fT2")) - p5[) as a functions of the arrival rates Xi, i = 1,2 and the staffing levels m,j, j — 1,2, as well as the 
state 7, where s is the time parameter as before. (We now will allow the parameters Xi and rrij to vary 
as well as the state.) With that new notation, we see that the frozen process is equal in distribution to 
the corresponding FTSP with new parameters, in particular, 

{D^(A?,m!?,r n ,s) : s > 0} = {D(X?/n, m?/n, T n /n, ns) : s > 0}, (47) 

with the understanding that the initial differences coincide, i.e., 

D(A?/n, m"/n, T n /n, 0) = Z>"(A™,m™, T n , 0). 

This can be checked by verifying that the constant transition rates are indeed identical for the two pro- 
cesses, referring to (|T2" j) -([T5 ]l and (|I3" ]) -(|33 |) . Since Af jn -> A i; i = 1,2 and m"/n -> rrij, j = 1, 2, by virtue 
of the many-server heavy-traffic scaling in ©, we will have the transition rates of D(Xf/n, rrij/n, T n /n, ■) 
converge to those of D(-y) = D(Xi, rrij, 7, •) whenever T n /n —> 7. 

7.2 Bounding Processes. We will use bounding processes in our proof of Theorem 16.11 and later 
results. We construct the bounding processes so that they have the given initial conditions at time and 
satisfy FWLLN's with easily determined continuous fluid limits, coinciding at time 0. We thus control 
the initial behavior. 

We first construct w.p.l lower and upper bounds for Z™ 2 . Recall that N*j, i,j — 1,2, are the 
independent rate-1 Poisson processes used in (|3Tj) . Let 

K(t) 

Lemma 7.1 For alln>l and t > 0, Z£(t) < Z£ 2 (t) < Zg(t) w.p.l. 

Proof. The bounding processes and are both initialized as Zf 2 (0) at time 0. They are defined 
in terms of the same rate-1 Poisson processes as Z" 2 , so that the three processes can be compared for each 
sample path. The lower-bound process Z% is the pure death process obtained by routing no new class- 1 
customers to pool 2 and letting all initial ones depart after receiving service. Hence, is decreasing 
monotonically to 0. The upper-bound process Z£ is a pure birth process obtained by having pool 2 not 
serve any of its initial class 1 customers and by assigning every server in pool 2 that completes service of 
a class 2 customer to a new class 1 customer, assuming that such customers are always available. Hence, 
Z^ is increasing monotonically to • The given process Z" 2 necessarily falls in between. □ 

We next construct w.p.l lower and upper bounds for Qf. The upper bound processes will have the 
specified arrivals but no departures, while the lower bound process will have no arrivals but maximum 
possible departures. Both processes will start at the initial values. 

Let Nf , Nfj and iV" be the previously specified independent rate-1 Poisson processes used in (|31l) . 
Let 

2 2 

Qlad) = Q-(0) -EE N tMdmp) - JV?(W(0))i), 

i=l j = l 

Q?, b (t) = Q?(P) + N?Wt), t>0. (49) 
Lemma 7.2 For all n > 1 and t>l, 



ZU°)~ N l2[^2 ^ Z n M ds 



(48) 
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Proof. Just as in Lemma lTH we get a w.p.l comparison because we construct both systems using the 
same rate-1 Poisson processes. The upper bound is immediate, because the two systems being compared 
have the same initial value and the same arrivals, but the upper bound system has no service completions 
or abandonments. For the lower bound, we separately consider the fate of new arrivals, customers initially 
in service and customers initially in queue. However, we allow the identity of departing customers to shift, 
which does not affect the result. In the lower bound system new arrivals never enter, so they necessarily 
leave sooner. Allowing for identity shift, customers initially in queue abandon as rapidly as possible in 
the lower bound system, since the rate is fixed at the initial rate 6*iQ™(0). Of course, some customers 
from queue may enter service. But all customers in service leave at least as quickly in the lower bound 
system because all servers are working continuously. In the lower bound system we act as if all servers in 
each pool are simultaneously serving customers of both classes. Hence, we do not need to pay attention 
to the service assignment rule. The identity of customers may change in this comparison, but the order 
will hold for the numbers. Hence the proof is complete. □ 

Unlike the processes in Corollary 16.11 we can easily establish stochastic-process limits for the as- 
sociated fluid-scaled bounding processes, and these continuous limits coincide at time 0. Let X™ b = 

{Ql,a> Q%ai %a> Ql,b> Qz,bi ^b)- 

Lemma 7.3 as n — > oo, X™^ x a _b in T)§, where x a ,fc = (<?i,<j) 92,0) z n , 

qi,b,Q2,b,Zb) is an element of Cq 

with 

Za(t) = Zl,2(0) - Ml, 2 / Z a (s) ds, 
JO 

z b (t) = Zi, 2 (0) + M2,2 / z b {s) ds, (50) 
Jo 

2 2 

qi,a(t) = q t (0) - X! X! i J, i<0 m t + 0i9i(°))*i 

qi,b{t) = qi(Q) + \it, t>0. (51) 



Proof. The stated convergence is a relatively simple application of the continuous mapping theorem. 
In particular, we first exploit the continuity of the integral representation, Theorem 4.1 in [32] . to establish 
the convergence (Z™, Z^) (z a , z ). The queue length bounds are simple linear functions. □ 

We will apply the bounding results above in i j8.ll starting with the proof of Lemma RTTl 



7.3 Rate Order for FTSP's. Given that we can represent frozen processes as FTSP's with ap- 
propriate state parameters 7, as shown in (|4T[) above, it is important to be able to compare FTSP's with 
different state parameters. We establish such a comparison result here for the FTSP in £13.21 using rate 
order. We say that one pure-jump Markov process Y\ is less than or equal to another Y2 in rate order, 
denoted by Y\ < r I2, if ah the upward transition rates (with same origin and destination states) are 
larger in Y% and all the downward transition rates (with same origin and destination states) are larger 
for Y\. The following lemma is an immediate consequence of the definition of rates in (fT2")) - (|15[) . 

Lemma 7.4 (rate order for FTSP's) Consider the FTSP in MMfor candidate states 7W = (q[ l) , q£> , z^ 2 ), 
i = 1,2. (a) 7/^1,2 > M2,2 and 

Jl) Jl)\ / I J2) (2) J2V . to3 



the 



(b) If Mi, 2 < M2,2 and 



th 



cn 



(-«r^ ; .^)<(-?r,9r.^) i" 

^(7 (1) r) <rD(jV\-). 

I (1) (1) / / (2) (2) (2). . 

{-q\ ,Qz ■~z\! 1 ) < (-q\ >,q\ ' ,-z\£) m 
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We will apply the rate order to get sample path stochastic order, involving coupling; see [TH1 W2\ . 
Ch. 4 ol [28] and §2.6 of [31]. We briefly discuss those bounds for a sequence of stochastic processes 
{Y n : n > 1}. We will bound the process Y n , for each n > 1, by a process i.e., for each n, we will 
establish conditions under which it is possible to construct stochastic processes and Y n on a common 
probability space, with Y^ 1 having the same distribution as Y n having the same distribution as Y n , 
and every sample path of YJ 1 lies below (or above) the corresponding sample path of Y n . We will then 
write Y£ < st (> s t)Y n - However, we will not introduce this "tilde" notation; Instead, we will use the 
original notation Y n and Y^ . As a first step, we will directly give both processes, Y n and Y" b ™ identical 
arrival processes, the Poisson arrival processes specified for Y n . We will then show that the remaining 
construction is possible by increasing (decreasing) the departure rates so that, whenever Y n = Y^ 1 , any 
departure in Y n also leads to a departure in Y™. That is justified by having the conditional departure 
rates, given the full histories of the systems up to time t, be ordered. 

When r\ t 2 — 1, rate order directly implies the stronger sample path stochastic order, but not more 
generally, because the upper (lower) process can jump down below (up above) the lower (upper) process 
when the lower process is at state or below, while the upper process is just above state 0. Nevertheless, 
we can obtain the following stochastic order bound, involving a finite gap. For the following we use the 
rational form r\,2 — j/k, j,k € Z + and the associated integer- valued QBD, as in §6 of [35] ■ (Recall 
Assumption 13.31 ) There is no gap when rj.,2 = I because then j = k = 1 and the jump Markov process 
and associated QBD process both are equivalent to a simple birth-and-death (BD) process. 

Corollary 7.1 (stochastic bounds from rate order for FTSP's) Consider the FTSP in ^3.21 with QBD 
representation based on n,2 = j/k as in §6 of |35j . for candidate states 7^ = (q[ l \ q%\ z[%), i = 1,2. 
(a) If fJ>i,2 > ^2.2 and 



then 

(b) If m,2 < [i-2,2 and 
then 



{-&\£\z$)<{-£\&\z?l) m M 3 , 



£(7 (1 V) < sf 5(7 (2, r)+(jVfc)-l. 

(-& a) ^ i) ,-*a)<(-& w ,^,-*S) - « 3 . 

^(7 (1) ,-) < si % (2 V)+(jVi)-l. 



Proof. We can do the standard sample path construction: Provided that the processes are on the 
same side of state in the QBD representation, we can make all the processes jump up by the same 
amount whenever the lower one jumps up, and make all the processes jump down by the same amount 
whenever the upper one jumps down. However, there is a difficulty when the processes are near the state 
in the QBD representation (which involves the matrix B for the QBD). When the upper process is 
above and the lower process is at or below 0, the lower process can jump over the upper process by at 
most (j V k) — 1, and the upper process can jump below the lower process by this same amount. But 
the total discrepancy cannot exceed (jVfc)-l, because of the rate order. Whenever the desired order is 
switched, no further discrepancies can be introduced. □ 

The complexity of the proof in Appendix [A] is primarily due to the fact that we allow general rational 
ratio parameters. If 7-1,2 = ^2,1 = 1, the proof can be much shorter, directly exploiting the sample path 
stochastic order in Corollary 1 7 . 1 1 above (where there is no gap). 

Remark 7.1 (rate order comparisons for queue difference processes) In the rest of this paper, in partic- 
ular in the proof of Lemma l8~3l in £18.11 Theorem 14. II in EfAland Theorem 14. 41 in £110. 4[ we will combine the 
results in this subsection and earlier subsections to establish rate order and sample path stochastic order 
comparisons between queue difference processes and associated frozen difference processes. A typical 
initial rate order statement will be of the form 

Di t2 <r D](T n ) in V([0,S}) (52) 

for some 5 > 0, which we now explain. First, Z?™ 2 is a function of the Markov process X$, which has 
state-dependent rates. Thus the "transition rates" of D\ 2 are understood to be functions of time t and 
Xg(t), the state of the Markov process X§ at time t, which includes the value of D™ 2 (t). However, 
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the right side of (|52")l is interpreted quite differently. We regard D r j(T n ) 1 conditional on the random 
state vector f, as a homogenous pure-jump Markov process constructed independently of Xq , with new 
rate-1 Poisson processes, as in H5.ll However, we deliberately construct the random fixed state vector 
r™ as a function of Xg in order to facilitate comparison of rates. Thus, DJ(T n ) and Xq, and thus D\ 2 , 
are dependent, but they are conditionally independent given the random state vector T n . Thus, the 
conclusion in (f52"|) means that the transition rates at each time t for each value of X^(t) are ordered. If 
the two processes are in the same state at some time t, then the two processes can make transitions to 
the same states, and each upward transition rate of D^(T n ) is greater than or equal to the corresponding 
upward transition rate of -D™ 2 , while each downward transition rate of D^(T n ) is less than or equal to 
the corresponding downward transition rate of D™ 2 - That rate ordering then allows the sample path 
stochastic order comparisons, as in Corollary 17. II 

7.4 A Sample Path Stochastic Order Bound. For the proof of Theorem l4.2l in SJTUJ we also need 
an upper bound process, unlike Q" b (£) in (|49|) . that does not explode as t — > oo. Hence, we now establish 
an elementary sample path stochastic order bound on the queue lengths that is stronger than the w.p.l 
upper bound in Lemma 17.21 Each of the two upper bound stochastic processes has the structure of the 
queue length in an M/M /oo model, with a service rate equal to the abandonment rate here, for which 
asymptotic results have been established |32) . 

Lemma 7.5 (sample path stochastic order for the queue lengths) For i = 1,2 and n > 1, let 

Qluit) = QUO) + N?(\?t) - Nr(6i I Qi M {s) ds), t > 0. 

Jo 

Then 

Qi,bd(t) Qi,bd(°°) as t^oo, 
where has a Poisson distribution with mean \f/0i and 

Q\bd = n ~ 1 Qi'ibd^ 1i,bd in T>([0, oo)) as n ^ oo, 

where qi.td evolves deterministically according to the ODE qi,bd(t) = Xi — 9iqi,bd(t), starting at qt^d(0) = 
<7i(0) for qi(0) part of x(0) in Assumvtion \'d.2\ Thus, 

qi,bd(t) < qi,bd(°°) = v {h/0i). 

Moreover, 

m,Q2)<st{Qlbd,Qlbd) in 2M[0,oo)). 

Proof. We apply Assumption 13.21 to get the intial queue lengths to converge. Just as for in 
Lemma 17.21 the upper bound system here provides no service completion at all. However, unlike the 
upper bound Q™ b in Lemma f7.2[ here abandonment from queue is allowed. Here we have sample path 
stochastic order because we can construct the two systems together, keeping the upper bound system 
greater than or equal to Q™(t) for all t. Whenever the constructed processes are equal, they can have the 
same abandonments, because the abandonment rate in both systems will be identical. □ 

7.5 Extreme- Value Limits for QBD Processes. In order to prove Theorem 16.11 in ^8 . 11 we 

exploit extreme-value limits for QBD processes. Since we are unaware of any established extreme-value 
limits for QBD processes, we establish the following result here. Recall that a QBD has states (i,j), 
where i is the level and j is the phase. If we only consider the level we get the level process; it is an 
elementary function of a QBD. 

Lemma 7.6 (extreme value for QBD) If C is the level process of a positive recurrent (homogeneous) QBD 
process (with a finite number of phases), then there exists c > such that 

lim P(||£|| t /logt>c) = 0. 

t— >oo 

Proof. Our proof is based on regenerative structure. The intervals between successive visits to the 
state (0, j) constitute an embedded renewal process for the QBD. Since the QBD is positive recurrent, 
these cycles have finite mean. Given the regenerative structure, our proof is based on the observation 
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that, if the process C were continuous real- valued with an exponential tail, instead of integer valued 
with a geometric tail, then we could establish the conventional convergence in law of \\C\\t — clogi to 
the Gumbel distribution, which implies our conclusion. Hence, we bound the process C above w.p.l 
by another process Cb that is continuous real-valued with an exponential tail and which inherits the 
regenerative structure of C. 

We first construct the bounding process Cb and then afterwards explain the rest of the reasoning. 
To start, choose a phase determining a specific regenerative structure for the level process C. let Si be 
the epoch cycle i ends, i > —1, with S—i = 0, and let L(n) be the set of states in level n. For each 
cycle i, we generate an independent exponential random variable Xi and take the maximum between 
C{t) and Xi for all Si-i < t < Si such that C(t) ^ L(0); i.e., letting {Xi : i > 0} be an i.i.d. sequence 
of exponential random variables independent of C and letting C(t) be the cycle in progress at time t, 
Cb(t) = C(t) V Xc(t)]-{c{t)<£L(o)}- Clearly, Lb inherits the regenerative structure of C and satisfies C < Cb 
almost surely. Moreover, by the assumed independence, for each x > and t > 0, 

P(C b (t) >x) = P{C{t) > x) + P(X > x) - P(£(t) > x)P{X > x), 

where X is an exponential random variable distributed as Xi that is independent of C(t). We now 
consider the stationary version of C, which makes Cb stationary as well. We let the desired constant c be 
the mean of the exponential random variables Xi. If we make c sufficiently large, then we clearly have 
P(Cb(t) > x) ~ e~ x l c as x — > oo, because the first and third terms become asymptotically negligible as 
x — V oo. (We choose c to make C(t) asymptotically negligible compared to X.) 

It now remains to establish the conventional extreme- value limit for the bounding process Cb- For that, 
we exploit the exponential tail of the stationary distribution, just established, and regenerative structure. 
There are two approaches to extreme- value limits for regenerative processes, which are intimately related, 
as shown by Rootzen [35]. One is based on stationary processes, while the other is based on the cycle 
maxima, i.e., the maximum values achieved in successive regenerative cycles. First, if we consider the 
stationary version, then we can apply classical extreme- value limits for stationary processes as in [27) . 
The regenerative structure implies that the mixing condition in [27] is satisfied; see Section 4 of [39] . 

However, the classical theory in [37] and the analysis in [35] applies to sequences of random variables 
as opposed to continuous-time processes. In general, the established results for stationary sequences in 
[27j do not extend to stationary continuous-time processes. That is demonstrated by extreme- value limits 
for positive recurrent diffusion processes in [8] [10]. Proposition 3.1, Corollary 3.2 and Theorem 3.7 of 
[8] show that, in general, the extreme- value limit is not determined by the stationary distribution of the 
process. 

However, continuous time presents no difficulty in our setting, because the QBD is constant between 
successive transitions, and the transitions occur in an asymptotically regular way. It suffices to look at 
the embedded discrete-time process at transition epochs. That is a standard discrete-time Markov chain 
associated with the continuous-time Markov chain represented as a QBD. Let N(t) denote the number 
of transitions over the interval [0, t]. Then Cb(t) = Cd(N(t)), where Cd(n) is the embedded discrete-time 
process associated with Cb- Since N(t)/t —> c' > w.p.l as t — > oo for some constant d > 0, the results 
directly established for the discrete-time process Dd are inherited with minor modification by Cb- Indeed, 
the maximum over random indices already arises when relating extremes for regenerative sequences to 
extremes of i.i.d. sequences; see p. 372 and Theorem 3.1 of [35J. In fact, there is a substantial literature 
on extremes with a random index, e.g., see Proposition 4.20 and (4.53) of [37J and also [1DJ. Hence, for 
the QBD we can initially work in discrete time, to be consistent with [37J[35]. After doing so, we obtain 
extreme-value limits in both discrete and continuous time, which are essentially equivalent. 

So far, we have established an extreme-value limit for the stationary version of Cb, but our process 
Cb is actually not a stationary process. So it is natural to apply the second approach based on cycle 
maxima, which is given in [35] [T] and Section VI. 4 of [2]. We would get the same extreme- value limit for 
the given version of Cb as the stationary version if the cycle maximum has an exponential tail. Moreover, 
this reasoning would apply directly to continuous time as well as discrete time. However, Rootzen [39] 
has connected the two approaches (see p. 380 of [39 ), showing that all the versions of the regenerative 
process have the same extreme-value limit. Hence, the given version of the process Cb has the same 
extreme-value limit as the stationary version, already discussed. Moreover, as a consequence, the cycle 
maximum has an exponential tail if and only if the stationary distribution has an exponential tail. Hence, 
we do not need to consider the cycle maximum directly. □ 
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8. Proofs of Three Theorems from Jj6]. In this section we prove the three theorems in 

8.1 Proof of Theorem 16.11 Theorem 16. II is an immediate consequence of the following three lem- 
mas: Lemmas 18.11 18.21 and 18.31 

Lemma 8.1 If 21,2(0) > 0, then, for all T > 0, P(mi < t < T Z^ 2 (t) > 0) -> 1 as n — !• 00. As a conse- 
quence, Z 21 =>• as n — > 00. 

Proof. By Assumption Z[ l 2 (0) =*> 21,2(0). From Lemma O we know that Z? 2 > Z% in 
D for all n > 1 w.p.l. From Lemma 17.31 we know that ^ z a in D as n -> 00, for 2 a in (|50|) . 
However, the integral equation for z a in (|50p is equivalent to the ODE i a (t) = ~[i\.iz a (t) with initial 
value z a (0) = £1,2(0). Since 21,2(0) > by assumption, it follows that z a (t) > z a (0)e _AI1 - 2 * > for all 
t > 0. Thus P(info< s <t Z™(s) > 0) — > 1 as n — > 00. Lemma [7.11 implies that the same is true for Z" 2 , 
which proves the first claim of the lemma. The second claim that Z 21 => as n — > 00 follows from the 
first together with the one-way sharing rule. □ 

Remark 8.1 (implications for one-way sharing rule) The conclusion of Lemma lS.ll reveals a disadvantage 
of the one-way sharing rule for very large systems. The lemma concludes that, for large n, if for some 
e > and to > Z™ 2 (io) > en , then Z^ 2 (t) is very likely not to reach for a long time, thus preventing 
sharing in the opposite direction, even if that would prove beneficial to do so at a later time, e.g., because 
there is a new overload incident in the opposite direction. 

In practice, we thus may want to relax the one-way sharing rule. One way of relaxing the one-way 
sharing rule is by dropping it entirely, and relying only on the thresholds fc™ 2 and k 2 1 to prevent sharing 
in both directions simultaneously (at least until the arrival rates change again) . Another modification is 
to introduce lower thresholds on the service processes, denoted by s™^, i 7^ j, such that pool 2 is allowed 
to start helping class 1 at time t if D 2 x > k 21 and Z™ 2 (t) < s™ 2 ) an d similarly in the other direction. 
We do not analyze either of these modified controls in this paper. 

Given Lemma IcTTl it remains to consider only the case 21.2(0) = 0. Hence, we assume that 21.2(0) = 
for the rest of this section. Here is the outline of the proof: We first prove (Lemmas 18.21 and 18.31 below) 
that Z 21 is asymptotically null over an interval [0,r], for some r > 0. We then prove that Zf 2 (i) must 
become strictly positive before time t in fluid scale. By Assumption 12.21 the optimal ratios for FQR-T 
satisfy r\^ 2 > r 2 ^. In Lemma I5~2l we consider the cases (i) x(0) E A U A + with r\_ 2 > r 2 ^\ and q\ (0) > 
and {%%) x(0) € S + ; in Lemma 18.31 we consider the remaining cases, i.e., a;(0) £ AU A + with n 2 = r 2.i or 
gi(0) = 0. Unlike the definition of D[\ 2 in ©, let D% tl be defined by 

rj 2 " 1 (t) = r2,ig 2 i (t)-Qi l (i), t>0. (53) 

Lemma 8.2 Assume that 21,2(0) = 0. If either one of the following two conditions hold: (i) x(0) G AUA + , 
f"l,2 > r 2,i and <7i(0) > 0, or (ii) x(0) G § + . then there exists t, < t < 00, such that 

lim P ( sup D 2 \(t) < ) = 1 
,wo ° \te[o,r] ' J 

for D% A in (USD, so that \\Z$ tl \\ T =*> as n -> 00. 

PROOF. We first show that the appropriate conditions hold in fluid scale at the origin. We start by 
assuming that x(0) € AU A+, which implies that di,2(0) = 5i(0) — ri,2#2(0) = 0. Since qi(Q) > by 
assumption, 32(0) > too. Since ri,2 > ^2,1 by assumption, ^2,1(0) = ^2, 192(0) — 9i(0) < ri, 292(0) — 
qi(0) = 0, so that we also have d2,i(0) < 0. If (ii) holds, so that a;(0) £ § + , then c?2,i(0) < by definition 
of§+ 

Given Assumption 13.21 we also have ^"(O) =^> x(0) in A U A + U S + . Hence, the fluid-scaled queueing 
processes converge to these initial values. In particular, we necessarily have D 21 (0)/n c?2,i(0) < as 
n — > 00. Hence, there exists c > such that P(D 21 (0) < —cn) — > 1 as n — > 00. Our goal now is to show 
that there exists t > such that P(sup 0<t<r D 21 (t) > 0) — > 0. That will imply the desired conclusion. 
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It only remains to show that the change in these quantities has to be continuous in fluid scale. For the 
purpose of bounding D 2 x — r 2 ,iQ 2 — Q™ above, it suffices to bound Q 2 above and Q™ below, as we have 
done in Corollary Hence < st D™ 21 = r 2 ^Q 2b -Q r { a . By Lemma [731 75™ 2,1 d 2)1 = r 2 ,iq2,b" 
qi. a , where the limit function d 2 i evolves continuously, starting with d 2 1 (0) = <i2,i(0) < 0. Hence there is 
a time r' such that d 2 i(t) < for all < t < r' . Asymptotically, by the FWLLN, the same will be true for 
the fluid scaled queue difference process 5" 2 1 . Hence, we deduce that P(sup 0<t<T {75™ 2 > 0) — ^ 
as n — > oo for any r with < r < r'. Since 75™ i < s t 75™ 2 ij the same is true for 75 2 1 . That completes 
the proof. □ 

The proof of Lemma 18.21 relies on a fluid argument since, under its assumptions and Assumption 
13.21 75™ 1 (0)/n converges to a strictly negative number as n -> oo. In particular, the difference 75 2 1 (0) 
without centering by k 2 i is order Op{n) away from the threshold k 21 . That fluid reasoning fails when 
r 2 ,i = ri,2 = r or when <7i(0) = since in either of these cases, gi(0) — ri^ 2 q 2 {Q) — qi(0) — r 2t iq 2 {Q) = 0. 
(By Assumption 13.21 (72(0) = if qi(0) = 0.) In these cases we will rely on the threshold k 2l to prevent 
class-2 customers to be sent to pool 1, and construct a finer sample-path stochastic-order bound for the 
stochastic system. Below we remove the centering by the threshold in 75™ 1 . 

Lemma 8.3 Assume that 21,2(0) = and that gi(0) — ^2,192(0) = {necessarily, x(0) £ S + , i.e., x(0) € 
A U A + ). In this case, there exists t, < t < 00, such that 

lim P\ sup DS tl (t) < k^A =1, 
™^°° \te[o,T] / 

for 75™ 1 m (|53|) . Hence, 1 1 ^2 1 1 1 r =^ as n -> 00. 

Proof. We start by showing that P(D 2 1 (0) < 0) — > I as n — > 00. That follows because, by 
Assumption 12.21 and the definitions © and (|53"|) . 

75^(0) = r2,iQJ(0) - Qf (0) < ri, a QJ(Q) - Qf (0) = -75", 2 (0) - k^ 2 . 

Assumptions 12.41 and 13.21 then imply that 75™ 2 (0) L and fc™ 2 — ^ 00 as n ~~ ^ 00 7 which together imply 
the initial conclusion. Going forward, it suffices to assume that we initialize by £'21(0) = 0- 

The rest of our proof follows three steps: In the first step, paralleling Lemma [7~4l and Corollarv l7.11 we 
construct a QBD that bounds 75™ 1 (without centering by k 21 ) in rate order, which enables us to obtain a 
stochastic order bound for D 21 ; see Remark 1 7. 1 1 The bound is constructed over an interval [0, r]. In the 
second step, we show how to choose r small enough so that the QBD bound is asymptotically positive 
recurrent. In the third step, we "translate" the QBD bound to a time-accelerated QBD, in the spirit of 
(H7J, and employ the extreme- value result for the time-accelerated QBD in Lemma \7. 61 to conclude the 
proof. 

Step One: We construct a stochastic-order bound for D 2 1; building on rate order. For r > 0, let 

X:(t) = (Ql b (t), Q 2 \ a {t), Z?(t)) and r™ = X? = (Q?, r , Q?, T , 2?), (54) 

where 

Ql,r = WQlbWr, Qlr = & QlaW V and EE ||Z™|| T . 

using the processes defined in (|48|1 and By Lemmas 17.11 and 1 7 . 2 [ for all n > 1, 

(Qi(t), -Q2 (*), Z?,a(*)) < (Q?,r. -Ga,r. Z " ) m K 3 for all t e [0, r] w.p.I. (55) 

As in jj7.ll let 75™ (r™) be the frozen difference process associated with 75™ 1 and T™ in (|54[) . (Recall that 
we are considering 75™ 1 here and not 75™ 2 .) However, to obtain positive results, we want to consider the 
process 75™ (r™) only for nonnegative values. We obtain such a process by working with the associated 
reflected process, denoted by 

B n f = Df*{T) = {D n y*(T,t) : t > 0}, Te E 3 

obtained by imposing a reflecting lower barrier at 0, where T specifies the fixed rates of 75"'* (r). We 
omit F from the notation for statements that hold for all r £ I3. The reflected process 75™'* is always 
nonnegative and has the same state space as the nonnegative part of the state space of D 21 . Within the 
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QBD framework used in §6 of [35] , we obtain the reflected process by omitting all transitions down below 
level 0; the specific QBD construction is given in Appendix [Bl 

It follows from (|55j) and Theorem 17.41 that we have rate order. By the analog of Lemma 17.11 there 
exists a constant K such that we have sample path stochastic order in 2?([0, r]), i.e., 

£>2,i <st D r }'*(r>) + K in £>([0,t]) for all n > 1, (56) 

for r; 1 in flM]). 

Step Two: We now show that we can choose r > so that there exist sets E n in the underlying 
probability space such that -Dj'*(r™) is positive recurrent in E n and P(E n ) — > 1 as n — >• oo. Paralleling 
(TXBT) (for the frozen process associated with -D™ 2 ), the set E n here is 

E n = {S,(T^) <0}, (57) 

where 5* is the drift for Df*. To find r > such that P(E n ) -> for £ n in ([57]), we analyze the 
asymptotic behavior of T™ = X™ in (|54"f . 

First, by Lemma [7731 X™ =>■ a;* = (91,6, q 2 ,a, Zb) in ©3 as n -> 00, where the components of x* are given 
in (1511) and x*(0) = x(0) by construction. Then, by the continuous mapping theorem for the supremum 
function, e.g., Theorem 12.11.7 in |45j . 

X" = X"/n => x T = (<7i, T , 92, t, z r) in R 3 , as n — t> 00, (58) 

where 

9i,t = Iki.illrj 92, r = hif g 2 ,a(*)V0 and z T = \\z b \\ T . 

0<t<r 

Since x(0) G A U A + by the assumption of the lemma, <5_(x(0)) > 0, where <5_ is the drift for the FTSP 
in (fl7|) associated with D™ 2 - 

For 7 ee 7 (t) ee (qi, b {t),q 2ta (t) V 0, *&(*)) let A r ( 7 ) ee A 2 , /t r ( 7 ) ee \x %2 (m 2 - « 6 (t)) + (<&,„(*) V 0), 
Ai( 7 ) ee /ii^mi + Mi,2£b(t) + 0iqi,b{t) and /ti( 7 ) = Ai. Let -D*( 7 ) be the reflected FTSP corresponding 
to -Dy'*(r™). The process £>*( 7 ) = {f*( 7 ,t) : i > 0} has upward jumps of size r 2 ,i with rate A r ( 7 ) and 

downward jumps of size r with rate /v( 7 )- It has upward jumps of size 1 with rate Ai( 7 ), and downwards 
jumps of size 1 with rate /ti( 7 ). By Theorem 7.2.3 in |26j . -D*( 7 ) is positive recurrent if and only if 
5* (7) < 0, where 

^( 7 )=r 2 , 1 (A r (7)-Ar(7)) + (Ai(7)-Ai(7)), 7 e K 3 - (59) 

Replacing 7 with 7t e i t , we have that D*( 7r ) is positive recurrent if and only if <S*( 7t ) < 0. Hence, it 
suffices to show that 5*(x r ) < for some r > 0. We do that next. 

We consider the two possible cases of the condition imposed in the lemma: (i) r 2 1 = r\,2 and (ii) 
9i(0) = 9 2 (0) = (with Zb(0) = 2i, 2 (0) = in both cases). First, in case (i), <5*(x*(0)) = -5_(x(0)) 
for S- (7) in (|TT|) . <5* (7) in ((59)) and 7 = x(0). Since we have already observed that £_(x(0)) > 0, we 
necessarily have 5*(x*(0)) < 0. 

In case (ii) with r*i )2 > r 2 ,i and 9i(0) = 9 2 (0) = (and again Zi i2 (0) = 0), <S*(x*(0)) = r 2i i(Ai - 
Mi.iTOi) + (A 2 — H 2 , 2 m 2 ), so that <5*(x*(0)) < if and only if (Ai — (ix^mx) + r 2i i(A 2 — ^ 2 , 2 to 2 ) < 0. 
However, this inequality must hold because, by Assumption 13.21 5_(x(0)) > 0, so that 5-(x(0)) = 
(Ai — jLti,imi) — ri j2 (A 2 — /i 2)2 m 2 ) > 0. Since r 2) i < ri. 2 , it follows that here too J*(x*(0)) < 0. 

Finally, the continuity of x* and <5*(x*) imply that we can find r > and 77 > such that 
Sup s£ p T ] 5*(x*(s)) < — ?y < 0. In particular, for that choice of r, 5*(x T ) < —77. Hence, P{E n ) = 
P(M r ?) < 0) -> 1, because <5*(X") => 5*(x*) in V and <5»(r™) =>■ <5*( 7t ) in M as n -> 00 by the 
continuous mapping theorem. 

S'iep Three: Finally, we apply the extreme- value result in Lemma 17.61 to the stochastic upper bound 
K + Dy*(T r ;) in for T™ in ([Ml)- For that purpose, observe that, paralleling (gTJ), {D n f ' (T",s) : s > 

0} = {D*(r™/n,ns) : s > 0} for D n f* and D* defined above, with r™/n x r in (5SJ). For the rest of 
the proof, we apply the Skorohod representation theorem to replace the convergence in distribution by 
convergence T"/n — > x T w.p.l, without changing the notation. 
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By the arguments above 

PCPSiJr/logn > c) < P{{K + \\D n f <*(T?) || T )l log n > c) = P((Jf + ||Z?*(T?/n)||„ T )/logn > c). (60) 

In order to apply Lemma 17.61 to the final term in (ISOl . we want to replace r™/n = X™ by a vector 
independent of n, say x e , such that D*(x e ) is positive recurrent and, for some no, X™ < ir e for all n > uq. 
Then we can apply Lemma 17.61 to get, for n > no, 

P((-K" + ||D*(r^/n)||„ T )/logn > c) < P((2X + ||D*(a; £ )|| nT )/logn > c) -> as n -> oo. 

That implies the claim because of the way the thresholds are scaled in Assumption 12.41 

We conclude by showing how to construct the vector x e such that D*(x e ) is positive recurrent and, 
for some n , T"/n = X" < x e for all n > n . If <?2,r > 0, then choose e such that < e < q2, T an d let 
a; e = (<?i, e , g2,e) 2 e ) = (gi jT + e, c/2,t — e, z b + e). Otherwise, let x e = (qi, r + e , 0, z b + e). Clearly x e < x T for 
all e > 0, so that {D*(x T , t) : < t < r} < si if + {£)*(a; £ , t) : < t < r} and, by the choice of r, we can 
find e > small enough so that S*(x e ) < 0, so that D*(x e ) is positive recurrent. □ 

8.2 Proof Theorem 16.21 Define the processes 

L? = Q? + 2?,i+2? )2 -mJ and L% = Q 2 l + Z^ + Z% 2 - m 2 \ (61) 

representing the excess number in system for each class. Note that {Lf) + = Q", i = 1,2. For all n 
sufficiently large, we will bound the two-dimensional process (Li,L 2 ) below in sample-path stochastic 
order by another two-dimensional process (L™ b ,L 2b ), n > 1. 

We construct the lower-bound process (L1 b ,L 2b ) by having £™ b (0) = L"(0) A fc" 2 , z = 1,2, and 
by increasing the departure rates in both processes L™ an d ma king it so that each goes down at 
least as fast, regardless of the state of the pair. First, we place reflecting upper barriers on the two 
queues. This is tantamount to making the death rate infinite in these states and all higher states. We 
place the reflecting upper barrier on L" at fc" 2 , where fc™ 2 > 0- This necessarily produces a lower 
bound for all n. By the initial conditions assumed for the queue lengths in Assumption 13 . 2 1 we have 
P((L?, 6 (0), L» 6 (0)) = (k'l 2 , fc£ 2 )) -j. 1 as n -> oo. 

With the upper barrier at fe™ 2 , the departure rate of L™(t) at time t > is bounded above by 
/ii^m" + fj,i >2 Z" 2 (t) + 0i*i2> based on assuming that pool 1 is fully busy serving class 1, that L™(t) 
is at its upper barrier, and that Z" 2 (t) agents from pool 2 are currently busy serving class 1 in the 
original system. With the upper barrier at fe™ 2 , the departure rate of L 2 (t) at time t is bounded above 
by fJ>2,2(™>2 ~ ^12^)) + $2*i 2; based on assuming that pool 2 is fully busy with Z™ 2 (t) agents from pool 
2 currently busy serving class 1, and that L 2 (t) is at its upper barrier fc™ 2 . Thus, we give L™ b and L 2 b 
these bounding rates at all times. 

As constructed, the evolution of (L™ b ,L 2 b ) depends on the process Z™ 2 associated with the original 
system, which poses a problem for further analysis. However, we can avoid this difficulty by looking at 
a special linear combination of the processes. Specifically, let 

U n = M2 ,2 (L? - kl 2 ) + Ml , 2 (i» - kl 2 ) and E/ b " = u 2 , 2 (^ fc - *?,„) + Mi.a^.ft " *? j3 )- (62) 

By the initial conditions for specified above By the established sample-path stochastic order 
(L™ b ,L 2b ) < st (L",L 2 ) and the monotonicity of the linear map in (l6"2"j) . we get the associated sample- 
path stochastic order U b <st U n . The lower-bound stochastic process U b has constant birth rate 
A b = /i2,2A™ + A*i,2A 2 and constant death rate 

EE U2, 2 (u M m™ + Ml,2^™2 CO + ^lfc" 2 ) + ^1,2(^2,2^2 ~ l"2,2 (*) ) + 02*1,2 ) 
= «2,2(«1,1 TO 1 + l k h) + Ml,2(«2,2m 2 l + 6> 2 fc^ 2 ). 

In particular, unlike the pair of processes (L™ b ,L 2 b ), the process U b is independent of the process Z™ 2 . 
Consequently, U b is a BD process on the set of all integers in (— oo, 0]. Since P((£™ b (0), £ 2 b (0)) = 
( fc ",2: fc i,2)) ~> 1 a s n -> oo, P(f/ b n (0) = 0) 1 as n oo. 

The drift in [/ b " is 



% ee AJ^ u b = U 2,2(A^ - m>x,i - 6 x kl 2 ) + ^(AJ - m> 2 ,2 - ^2^2)- 



(63) 
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Hence, after scaling, we get 8 b /n — > 8 (recall that k™ 2 is °( n )), where 

$b = M2,2(Ai - mxfxi^) + Ati,2(A 2 - m 2 p 2j2 ) > 0, (64) 

with the inequality following from Assumption 13.11 

Now we observe that — U b is equivalent to the number in system in a stable M/M/l queueing model 
with traffic intensity p" — > p„ < 1, starting out empty, asymptotically. Let Q* be the number-in-system 
process in an M/M/l system having arrival rate equal to A» = /i2,2Mi,i™± + Mi, 2/^2,2^2, service rate 
/i» = fJ-2.2^1 + Mi, 2^2 and traffic intensity = A*/p* < 1. Observe that the scaling in U b is tantamount 
to accelerating time by a factor of order 0(n) in Q». That is, {— f/"(t) : i > 0} can be represented as 
{Q*(c„£) : i > 0}, where c n /n — » 1 as rt — > 00. Wc can now apply the extreme- value result in Lemma [7751 
for the M/M/l queue above (since the M/M/l BD process is trivially a QBD process) to conclude that 
||Q*||t = Op(log(i)). This implies that —U£/\og{n) is stochastically bounded. 

From the way that the reflecting upper barriers were constructed, we know at the outset that L™ b (i) < 
fc™ 2 an d L'2 b (t) < fc" 2 . Hence, we must have both L" b — k™ 2 and L 2 b — fc™ 2 non-positive. Combining 
this observation with the result that (—[/")/ log n is stochastically bounded, we deduce that both (fc™ 2 ~~ 
L™ 6 )/logn and (fc" 2 — L 2 6 )/logn are stochastically bounded, i.e., the fluctuations of L™ b below fc" 2 are 
Op (log n). The result follows because — Lf < st —L™ b , i = 1,2, and from the choice of fc" 2 , which satisfies 
fc" 2 /logn — > 00 as n — > 00 by Assumption 12.41 

8.3 Proof of Theorem 16.41 We now show that the interval [0, r] over which the conclusions in <J5] 
are valid can be extended from [0, r] to [0, 00) after Theorem 14. II has been proved over the interval [0, r]. 

For this purpose, we use the processes L" and L 2 defined in (IBTl) . By Lemma [8TT1 we only need to 
consider the case 21,2(0) = 0. By Lemmas 18.21 and 18.31 there exists r > such that 

hm P(\\DZ 1 \\ T <kZ 1 ) = l. 

Hence, the claim of the theorem will follow from Lemma 18.11 if we show that for some to satisfying 
< to < r it holds that 21,2(^0) > 0, where z\,i is the (deterministic) fluid limit of Z" 2 as n — > 00, which 
exists by Theorem 14. II We will actually show a somewhat stronger result, namely that for any < e < 8, 
where 8 is chosen from Lemma [A . 1 1 and thus Theorem 14. II there exists to < e such that ^1,2(^0) > 0. We 
prove that by assuming the contradictory statement: for some < e < 8 and for all t € [0, e], £1,2 (*) = 0. 

By our contradictory assumption above, Z" 2 = op(l), i.e., ||Z" 2 /rt|| e =>- 0. Recall also that 
Z21 — op(l) over [0, e] (since e < r, and r is chosen according to Lemmas 18.21 and I8.3|) . Hence, by 
our contradictory assumption and by our choice of e, there is negligible sharing of customers over the 
interval [0, e]. We can thus represent L 1 / in (|6"Tj) . i — 1, 2 by 

L?(t) = L?(0) + N?(\?t) - N* ti L iti J\l?(s) A 0) ds^j - U j\^( S ) V 0) <h\ + o P {n), (65) 

for i = 1, 2 and < t < 8 a,s n — > 00, where iVf , Nfi and iV" are independent rate-1 Poisson processes. 
The op(n) terms are replacing the (random-time changed) Poisson processes related to Z™ 2 and Z%i, 
which can be disregarded when we consider the fluid limits of (|65p . The negligible sharing translates into 
the Op(n) term in (|65[) by virtue of the continuous mapping theorem and Gronwall's inequality, as in §4.1 
of [3J. 

Letting L™ = Lf/n, i — 1,2, and applying the continuous mapping theorem for the integral repre- 
sentation function in (|65l) . Theorem 4.1 in [32], (see also Theorem 7.1 and its proof in [3"2~]), we have 
(Z™,L 2 ) (Li,L 2 ) in P([0,e]) as n ->• 00, where, for i = 1,2, 

L<(t) = Li(0) + (Ai - Mi,imi)t - / \pn,i{Li{s) A 0) + 0i(L t -(«) V 0)] ds, < i < e, 

Jo 

so that 

£J(t) = = (Ai - Mi,i m i) - Vi,i{L t {t) A 0) - ( (£ 4 (t) V 0), < t < e. 

By Assumption 13.21 both pools are full at time 0, so that Lj(0) > 0. Moreover, for j = 1,2, L\ = 
(Ai — Hi,i)/9i is an equilibrium point of the ODE L-, in the sense that, if Li(trj) = , then Zi(t) = L\ 
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(66) 



for all t > to. (That is, L\ is a fixed point of the solution to the ODE.) It also follows from the derivative 
of Li that Li is strictly increasing if L^(0) < Lf, and strictly decreasing if Li(0) > Lf, i — 1,2. 

Recall that p\ > 1, so that Ai — /xi^mi > 0. Together with the initial condition, Li(0) > 0, we see 
that, in that case, L\{t) > for all t > 0. First assume that p2 > 1 . Then, by similar arguments, 
£>2(t) > for all t > 0. In that case, we can replace Li with qi(t) = {Li{t)) + , i = 1,2, where qi is the 
fluid limit of Q" over [0, e]. We can then write, for t G [0, e], 

qi(t) = <?i(0) - (Ai - ^ M mi)£ - 6>i / qi(s) ds, 

Jo 

<h{t) = 92(0) - (A 2 - H2,2m 2 )t - 9 2 / q 2 {s) ds, 

Jo 

so that, for t G [0, e], 

di, 3 (t) = 9? + (9i (0) - g?)e~ eit - r (qg + (q 2 (0) - q^e^) 
= (<?l ~ rq a 2 ) + (ft (0) - Oe- 9lt - r(q 2 (0) - S^e"" 2 '. 
First assume that x(0) G A U A+, so that di, 2 (0) = 0. From ([SBl . 

d' h2 (t) ee ld 1>2 (t) = -fl!( gi (0) - qDe^ + r6 2 (q2(0) - q^e^ . 

Hence, d[ 2 (0) = Ai — /xi.imi — ^igi(O) — r(A 2 — /i 2 , 2 ) +?'0 2 (7 2 (O). If follows from (fTT)) and the assumption 
21,2(0) = 0, that d' 12 (0) = S-(x(0)). By Assumption, x(0) G AU A+, so that d' 1:2 (0) = 5_(x(0)) > 
(that follows from the definition of A and A + in (fT8|) and (|19|) . and the fact that <5_ > <5+). Hence, di j2 
is strictly increasing at 0. Now, since rfi j2 (0) = 0, we can find t\ G (0, e], such that d\^ 2 (i) > for all 
< t < t\. This implies that F(inf <t<t 1 D™ 2 (t) > 0) -> 1 as n -> 00. The same is true if a;(0) G § + . 

Since ||Z™ 2 /n|| e ^> 0, as a consequence of our contradictory assumption, it follows from the represen- 
tation of Z™ 2 in Theorem 15.11 that 

Z? i2 (t) = n _1 iV| i2 (p2,2 m 2t) + op(l) as n -> 00. (67) 

However, by the FSLLN for Poisson processes, the fluid limit z 12 of Z n in (|6T|) satisfies Zi t2 (t) = /i 2i2 m 2 i > 
for every < t < t\. We thus get a contradiction to our assumption that Z\ <2 .if) = for all t G [0, e]. 

For the case p 2 < 1 the argument above still goes through, but we need to distinguish between two 
cases: Z 2 = and i 2 > 0. In both cases L 2 is strictly decreasing. In the first case, this implies that i 2 is 
negative for every t > 0. It follows immediately that qi(t) — rq 2 (t) > for every t > 0. If i 2 (0) > 0, then 
necessarily £i(0) > 0, and we can replace Li with qi, i = 1,2, on an initial interval (before L 2 becomes 
negative). We then use the arguments used in the case p 2 > 1 above. 

9. Completing the Proof of Theorem I4.lt Characterization. The C-tightness result in The- 
orem [52] implies that every subsequence of the sequence {(Xg,Y^) : n > 1} in (|24l) has a further 
converging subsequence in P14QO, 00)), whose limit is in the function space C14QO, 00)). To establish the 
convergence of the sequence {(Ag,F 8 ™)} we must show that every converging subsequence converges to 
the same limit. 



Corollary 16. 31 and Theorem 16.31 imply that we can simplify the framework over an initial time interval 
[0, t]. In particular, it suffices to focus on the sequence {X n } in 2? 3 ([0,r]) given in (|1T) , where the limits 
of the subsequences will be in Cs([0,t]), but we will later show that this restriction can be relaxed. In 
particular, we will show that convergence holds in 2?i4([0, 00)) 

We achieve the desired characterization of X n in 2?3([0, r]) by showing that the limit of any converging 
subsequence is almost surely a solution to the ODE (1221) over [0, r]. The existence and uniqueness of the 
solution to the ODE has been established in Theorem 5.2 in 35 . As indicated in £14.41 we have developed 
two different proofs, the first proof exploits random measures and martingales, following [T71 [2Sj; it is 
given here in § £19.21 and 19.31 The second exploits stochastic bounds as in the proof of Lemma [8.31 it is 
given in Appendix |XJ Both proofs start from the following subsection. 

9.1 Proof of Theorem I4.lt Reduction to Integral Terms. Let X be the limit of a converging 
subsequence of {X n : n > 1} in fl4"Tl) in L) ([0,r]) for r in Theorem 16.31 We consider n > 1 with the 
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understanding that the limit is through a subsequence. Many of the terms in pTj) converge directly 
to their counterparts in (|23[) because of the assumed many-server heavy-traffic scaling in ^2.31 and the 
convergence X n => X through the subsequence obtained from the tightness. Indeed, the only exceptions 
are the integral terms involving the indicator functions. Let I™ it l q % 1 i5 and 7™ 2 i be the i th integral term 
in the respective expression for Z" 2 , Qi and Q 2 m (|4T|) . i — 1,2. We first observe that these sequences 
of integral terms are tight. 

Lemma 9.1 {tightness of integral terms) The six sequences of integral processes : n > 1}, {Iq \ j : 
n > I}, and {I" 2i '■ n — 1} involving the indicator functions appearing in (|4ip are each C-tight in 
P([0,r]). 

Proof. We consider only the integral term Ign, because the others are treated in the same way. 
First, boundedness is elementary: < Ig ± 1 (t) < tm^/n. Second, the modulus in (|32|) is easily controlled: 
w{Il l l X,T)<C,m n /n^ C,m 2 . - Q 

Hence, we can consider a subsequence of our original converging subsequence in which all these integral 
terms converge to proper limits as well. Hence we have the following expression for X, the limit of the 
converging subsequence. 



Zi, 2 (t) = zi, 2 (0) + M2,24,l(t) " ftlfllz,*® 



Qi(t) = <?i (0) + Mt - mit - ni^I q ,iA(t) - ^2,2^1,2^) - 0i / Qi(s) ds, 



n 



Q*{t) = <Z2(0) + \ 2 t - /i2,2/g,2,l(t) " Ml,2^,2,2(*) - 2 / Q 2 (s)) ds 





In (|68p . we have exploited the assumed convergence of the initial conditions in Assumption 13 . 21 to replace 
X(0) by x(0) in (|fJ51) . At this point, it only remains to show that the terms I z> i, Iq,\,% and I q ,2,i appearing 
in (|68p necessarily coincide almost surely with the corresponding terms in the integral representation (|23[) 
associated with the ODE in (|2"2"|) over the interval [0, r]. That will uniquely characterize the limit over 
that initial interval [0,r] because, by Theorem 5.2 of [35], there exists a unique solution to the ODE. 

Thus, it suffices to establish the following lemma, which we do in two different ways, one in the next 
two sections and the other in Appendix [A] 

Lemma 9.2 (representation of limiting integral terms) For r in Theorem \G.'3[ the integral terms in (I68[) 
necessarily coincide with the corresponding integral terms in (|23p with X substituted for x for < t < t, 
e.g., ^ 

4n(*)= / Ki,2(X(s))Z h2 {s))ds, 0<t<r, w.p.l. (69) 
Jo 

9.2 Exploiting Random measures. For the rest of our proof of Lemma 1^21 we closely follow Hunt 
and Kurtz |17j . which applies Kurtz [25] . There are two key steps: (i) exploiting random measures and 
(ii) applying a martingale representation to characterize the limit in terms of the steady-state distribution 
of the FTSP in i)3.2l This subsection is devoted to the first step; the next subsection is devoted to the 
second step. 

We now introduce random measures in order to expose additional structure in the integral term I qi i } i 
in The random measures will be defined by setting 

P n ([0, t]xB)= f l {D » aW6B} ds, t > 0, (70) 
Jo 

where B is a measurable subset of the set E of possible values of D™ 2 . Given the definition of D" 2 m @ i 
we exploit Assumption 13.31 stating that r\ t % is rational and the assumption after it that the thresholds 
are rational as well, so that we can have the state space E be discrete, independent of n. (This property 
is not necessary for the analysis at this point, but it is a helpful simplification. The space E could even 
be taken to be a subset of Z, using the construction in §6 of [3S] by renaming the states in E.) Of course, 
we are especially interested in the case in which B is the set of positive values; then we focus on the 
associated random variables i/ n ([0,t} x (0, oo)), n > 1, as in (|1T|) . 
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As in [17U25) . it is convenient to compactify the space E. We do that here , first, be adding the states 
+00 and —00 to E and, second, by endowing E with the metric and associated Borel cr-field from K 
induced by the mapping ip : E — > [—1, 1], defined by ip(x) — x/(l + \x\). That makes E a compact metric 
space. We then consider the space M = M(S) of (finite) measures fi on the product space S = [0, 8] X E 
for some 5 > 0, such that fi([0, t]x E) = t for all t > 0. Moreover, we endow M with the Prohorov metric, 
as in (1.1) of |25j . Since S is compact, the space M inherits the compactness; i.e., it too is a compact 
metric space, by virtue of Prohorov's theorem, Theorem 11.6.1 of |45j . 

Let V = V(M) = 7 5 (Af(S')) be the space of probability measures on M(S), also made into a metric 
space with the Prohorov metric, so that convergence corresponds to the usual notion of weak convergence 
of probability measures. As a compact metric space, M(S) is a complete separable metric space, so that 
this is a standard abstract setting for weak convergence of probability measures [6l[TTJ[45]. By Prohorov's 
theorem, this space V of probability measures on M(S) also is a compact metric space. 

Thus we have convergence of random measures v n v as n — > 00 if and only if E[f(v n )] — > E[f(v)] as 
n — > 00 for all continuous bounded real- valued functions / on M . On the other hand, by the continuous 
mapping theorem, if we have v n =>■ v as n — ¥ 00, then we also have f{y n ) =>■ f(y) as n -> 00 for each 
continuous function on M. One reason that the random measure framework is convenient is that each 
continuous bounded real-valued function / on S corresponds to a continuous real-valued function on 
M(S) via the integral representation 

f{p)= [ f(s)dfi(s). 
As a consequence, if v n v as n — > 00, then necessarily also 

/ f(s)dv n (s)^ / f(s)dv(s) in R as n ->• 00 
Js Js 

for all continuous bounded real-valued functions / on S. 

In our context it is important to observe what are the continuous functions on E after the compactifi- 
cation above. The new topology requires that the functions have finite limits as k — > +00 or as k — > —00. 
All the functions we consider will be continuous on E because they take constant values outside bounded 
sets. We will use the following stronger result, which is a special case of Lemma 1.5 of [2"5] . 

Lemma 9.3 (extended continuous mapping theorem) If f is a continuous bounded real-valued function on 
S and {/„ : n > 1} is a sequence of measurable real-valued functions on S such that \\f n — f\\s — > as 
n — > 00 ; then 

/ f n (s)dv n (s)^> / f(s)dv(s) in R as n->ac. 
Js Js 

Proof. First, starting from the convergence v n v, apply the Skorohod representation theorem to 
obtain versions converging w.p.l, without changing the notation. Then, by the triangle inequality, 

I / f n (s)du n (s)- f f(s)du(s)\< \ f f n (s)dv n (s) - I f(s)dv n (s)\ + \ f f{s)dv n {s) - f f(s)dv(s)\ 
Js Js Js Js Js Js 

<\\fn-f\\sv n (S) + \ f f(s)dv n ( S )- [ f(s)dv(s)\^Q asrwco, 
Js Js 

using the uniform convergence condition and the limit v n (S) — > v{S) < 00 in the first term and the 
continuous mapping in the second term. This shows convergence w.p.l for the special versions and thus 
the claimed convergence in distribution. □ 

The random measures we consider are random elements of M(S) , where S is the product space [0, 5} x E. 
These random measures can be properly defined by giving their definition for all product sets [0,t] x B, 
as we have done in ([701) . The usual extension produces full random elements of M(S). 

We exploit the compactness of E introduced above to obtain compactness of 'P(M) and thus relative 
compactness of the sequence {(^j 1 , l/n ): n > !}• 

Lemma 9.4 (relative compactness) The sequence {(Xg ,v n );n > 1} defined by (|24[) and (|T0[) is relatively 
compact in T>q([0, 6]) x M(S) for any 6 > 0. 
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Proof. We already have observed that, because of the compactness imposed on E, the space 
V(M(S)) is compact. By Theorem 15. 21 the sequence {X" 2 ■ n > 1} is tight and thus relatively compact. 
However, relative compactness of the components implies relative compactness of the vectors. Thus the 
sequence {(Xg, v n );n> 1} defined by {25) and (70)) is relatively compact in V 6 ([0, 6} x M (5). □ 

Another crucial property of the random measures on the product space is that the random measures 
themselves admit a product representation or factorization, as indicated by Lemma 1.4 of |25j ; also see 
Lemma 2 of [17]. This result requires nitrations. For that, we observe that Xq is a Markov process 
and v n restricted to [0,t] x E is a function of the Markov process X§ over [0,t]. Thus, we can use the 
nitrations J 7 ™ generated by Xg for each n > 1. In our context, we have the following consequence of 
Lemma 1.4 of [25]. 

Lemma 9.5 (factorization of the limiting random measures) Let (X,v) be the limit of a converging subse- 
quence of {(Xq, v n ); n > 1} in T>^([Q 1 S\) x M(S) obtained via Lemma \9A\ Then there exists p s = p s (B), 
a measurable function of s for each measurable subset B in E and a probability measure on E for each s 
in [0,5], such that, for all measurable subset B\ of [0, 5} and B 2 of E, 

v(B 1 xB 2 )= f Ps {B 2 )ds. (71) 

As a consequence of the three lemmas above, we obtain the following preliminary representation. 

LEMMA 9.6 [initial representation of limiting integral terms) Every subsequence of the sequence 
{(Xq ,v n );n > 1} defined by (|24[) and (1701) in T>e([0,8]) x M(S) as a further converging subsequence. 
Let (X,is) be a limit of a convergent subsequence. For any 6 < r for r in Theorem 16.31 the integral 
terms in (|68l) necessarily coincide with the corresponding integral terms in (|41[) with a probability pi t2 (s) 
substituted for l{,Df 3 (s)>o} an d X substituted for x for <t < 8, in particular, 

4i,i(*)= / PiAs)Z h2 (s))ds, 0<t<8, w.p.l. (72) 
Jo 

where Z\ t2 is the component of X and Pi^(s) = p s ((0, oo)) for p s in (|7ip . so that pi -2 (s) is a measurable 
function of s with < pi.2(s) < 1, < s < 6. 

Proof. By Lemma |9.4[ we are justified in focusing on a converging subsequence with limit (X,v), 
where X = (Qi,Zij). For (fT2"j) . we focus on (Z\, 2 ,v). As before, apply the Skorohod representation 
theorem to obtain a version converging w.p.l along the subsequence, without changing the notation. For 
the corresponding terms indexed by n, 

= / ZiMhD? 2 {s)>0}ds = f Zl 2 {s) v n (ds x dy), (73) 
Jo ' Jo 

so that we can apply Lemma \9. 31 to deduce that 

Igi i(f) -> 4i,i(t) = Mi.2 / Z h2 (s) v(ds x dy). 

Jo 

Finally, we apply Lemma 19.51 to show that the representation of v in ([73| is equivalent to (l72l . □ 

It now remains to determine the term pi^{s) in the integrand of the integral (|T2[) . In the next section 
we will show that we can write p\^{s) = P(D(X(s), oo) > 0), thus completing the proof of Lemma |9~21 

9.3 A Martingale Argument to Characterize the Probability in the Integrand. We now 

finish the proof of Lemma 19.21 by characterizing the probability measure p s in Lemma 19.51 

Proof of Lemma |9~2"1 We will prove that P\,%{s) = P(D(X(s), oo)) > for almost all s in the 
integral (|72l) . where D(X(s), oo)) is a random variable with the steady-state distribution of the FTSP 
in £)3.2I depending on the state X(s), which is the limit of the converging subsequence of the sequence 
{X n ; n > 1}. This step will make (JJ^J reduce to the desired (|6"9"|) . 

We first comment on the exceptional sets. We establish the result w.p.l, so that there is an exceptional 
set, say T, in the underlying probability space O with P(Y) = 0, such that we claim the conclusion of 
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Lemma 19.21 holds in ft — T. However, for each u E il — T, we find an exceptional set \l/(w) in [0, 6] 
where the Lebesgue measure of ^(lj) is 0. However, the integral in Lemma 19.21 is unchanged if we 
change the definition of the integrand on a set of Lebesgue measure 0. Hence, we can assume that 
Pi, 2(s) = P(D(X(s), 00) > 0) for all s in [0,S] for each sample point cj e £1 — T. After doing that, we 
obtain the w.p.l conclusion in Lemma 19.21 

We remark that it is possible to obtain a single exceptional set $ in [0,6] such that pi t 2(s) = 
P(D(X(s), 00)) > for all s 6 [0,(5] — <!/ w.p.l. The construction to achieve that stronger goal is 
described in Example 2.3 of Kurtz [25] on p. 196. The conditions specified there hold in our context. 
Since that property is not required here, we do not elaborate. 

Continuing with the main proof, we now aim to characterize the entire probability measure p s on 
E appearing in Lemma 19.51 We will do that by showing that p s satisfies the equation characterizing 
the steady-state distribution of the FTSP D(X(s), •) for almost all s with respect to Lebesgue measure 
(consistent with the notion of an averaging principle). Since the FTSP D(X(s), •), given X(s), is a CTMC 
with the special structure (only four transitions possible from each state, and only two different cases 
for these, as shown in (|12[) -([T5 |) ). just as for finite-state CTMC's (in elementary textbooks), it suffices to 
show that 

$>.({*})Qij(*(«))=0 for all j (74) 



for almost all s in [0, 6] with respect to Lebesgue measure, where i and j are states of the FTSP and 
Qij(X(s)) in ([74]) is the (i,j) th component in the infinitesimal rate matrix (generator) of the CTMC 
D(X(s),-). 

However, we will follow [TT] and use the framework in §§4.2 and 8.3 of [TTJ. In particular, the FTSP 
satisfies the assumptions in Corollary 8.3.2 on p. 379 in [TT] . As in (9) of [T7J, this step corresponds 
to an application of Proposition 4.9.2 of [TT], but the simple CTMC setting does not require all the 
structure there. Following the proof of Theorem 3 in [TTJ (and §2 of [25]), we now develop a martingale 
representation for /(-D™ 2 )> where / is a bounded continuous real- valued function on the state space E of 
D™ 2 . This construction is the standard martingale associated with functions of Markov processes, just 
as in Proposition 4.1.7 of [TTJ. Since, D" 2 is a simple linear function of the CTMC Xg in (j6]), we can 
write /(-D™ 2 ) = ff(Ag ) for some continuous bounded function g. The martingale will be with respect to 
the filtration J 7 ™ generated by the Markov process X£ (as in Lemma [ 



In particular, using the martingales in (1401) . and rearranging terms so that the martingale terms appear 
on the left-hand side, we conclude that, for any bounded continuous function / on E, the following is an 
^"■-martingale : 



M; = f(Dl 2 (t))~f(Dl 2 (0))~ ^ \?[f(DZ 2 (s-) + l)-f(Dl 2 (s-))]ds 
W + 0iQUs))[f(DZ 2 (s-) - 1) - f(Dl 2 (s-))]ds 
hD? 2 (s)>oM,2Zl 2 (s) +H2,2{m n 2 - Zl 2 { S )))[f{Dl 2 { S -) - 1) - f(D« 2 (s-))]ds 







(i 



1 hD l2 (s )m (vi,2Zl 2 (s) + V2,2(m2 - Zl 2 {s)))[f{Dl 2 {s-)+r) - f(D^{ 8 -))\da 



hD l2[ s)>o } e2Q n 2{s)[f{Dl 2 ( S -) +r) - f(Dl 2 ( S -))}ds 

Wf(Dl 2 (s-) - r) - f(Dl 2 (s-))]ds. (75) 







By Lemma \G. II and the continuity of addition at continuous limits, = MJ/n => Oe in T> as n — > 00, 
for MJ in ([75]). In addition, we have n' 1 (f (D^ 2 (t)) ~ f{D\\ 2 (0))) => Oe in V since / is bounded. We 
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write the remaining terms of as 

D](t) = M?(t) n-\f(D\\ 2 (t)) /(£T >2 (0))) = / Klf(y + 1) - f(y)]u n (ds x dy) 

J(0,t)xE 

-I (m n 1 + #iQ?(s))[/(y - 1) - f(y)K(ds x dy) 

J(0,t)xE 

- [ l {y> oM, 2 Zl 2 (s) + /i 2 , 2 (m? - Zl 2 {s)))[f{y - 1) - f{y)]v n {ds x dy) 

J(0,t)xE 

l {y < 0} (^, 2 Zl 2 (s) + M 2, 2 (m? - Zl 2 {s)))[f{y + r) - /(y)]i/ n (da x dy) 

(0,t)xB 

- / l{ y >o}02<5S(*)[/(i/ + r) - f(y)]v n {ds x dy) - / ^[/(y - r) - /(i/)]^ n (d S x dy). 

J(0,t)xE J(0A)xE 

By Lemmas 19.61 and 10 . 31 £)* Df = Oe as n — > oo along the converging subsequence, where 

5/ = / Xi[f(y + 1) - /fo)Mdn x dy) - / (m x + 0iQi(s))[/(y - 1) - /(y)M^ x dy) 

J(0,t)xB 4 J(0,t)xE 

1{ v >o}{Vi,2 z iA s ) + mA m 2 ~ ^i,2(s)))[/(y - 1) - f{y)}v{ds x dy) 

(0,i)xE 

1{2/<0}(mi,2^i,2(s) + H2,2( m 2 - Zi, 2 (s)))[f(y + r) - f{y)}v(ds x dy) 

(0,t)xE 



l {v >o}02Q2(s)[f(y + r) - f(y)Hds x dy) - / A 2 [/(y - r) - f(y)]v(da x dy). 

>(0,t)xE J(0,t)xE 

(76) 

However, just as in [17], we can identify the limit Df in ((76)) as the integral with respect to the random 
measure v of the infinitesimal generator of the FTSP D(X(s),-) applied to the test function /. In 
particular, for each sample point in the underlying probability space f2 supporting (X, v) except for a 
subset T with P(Y) = 0, from Lemma 1^51 we obtain 

/ [ [Q(X(s))f}(y)p s (dy)ds = for all t > 0. (77) 
Jo Je 

where [Q(X(s))f](y) is the generator of the CTMC D(X(s), ■) applied to / as a function of y in E. As 
a consequence, 

/ [Q{X(s))f](y)p s (dy) = for almost all s with respect to Lebesgue measure. (78) 
Je 

It follows from Proposition 4.9.2 page 239 in [TT], that p s is the (unique) stationary distribution of the 
FTSP D(X(s), ■) for almost all s. (This step is equivalent to (|74p.) We apply Lemma [3.11 to conclude 
that the FTSP D(j, •) has a unique stationary distribution on E for all 7 G §. □ 

9.4 Proof of Theorem (47TJ We can now summarize the proof of our main result - the FWLLN via 
the AP. 

Proof of Theorem 14. H There are two steps: (i) establishing convergence over an initial interval 
and (ii) expanding the interval of convergence. 

(i) Establishing convergence over [0,r]. By Theorem 15.21 the sequence {{X§,Y%) : n > 1} in 
is C-tight in 2?u([0, 00)). By Theorem 16.31 there exists r > such that the limit point of a converging 
subsequence of X£ in D([0,r]) is also the limit point of X$'* in ©, whose representation is specified 
in (l37| - ([39)) . Thus, it suffices to next characterize the limit of a converging subsequence of the sequence 
{A n } = {l/n(Qi, Q 2 , Z r i 2 )} in (l4"TT) over an interval [0, r]. With that characterization complete, we 
obtain the full convergence (Xg,Yg) =$> (x 6 ,y 8 ) in 2?i 4 ([0,t]) directly, exploiting Theorem 16.31 By 
Lemma [9.21 above, we complete the characterization step, showing that P(X = x) = 1 in T>s([0, r]), 
where a; is a solution to the ODE in (|22l) with the initial condition x(0) specified by Assumption! 



(ii) Expanding in the interval of convergence. After establishing the convergence over an initial interval 
[0,t], we can apply Theorem 16.41 f which uses Theorem 14.11 over [0,r]) to conclude that any limit point 
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of the tight sequence Xg is again a limit of the tight sequence Xq '* in © over the entire half line 
[0,oo), showing that r places no constraint on expanding the convergence interval. Moreover, by part 
(ii) of Theorem 5.2 in |35j . any solution to the ODE, with a specified initial condition, can be extended 
indefinitely, and is unique. Hence that places no constraint either. Finally, the martingale argument 
allows us to uniquely characterize the steady-state distribution of the FTSP D(<y, •) in H3.2I even when 
the state 7 is not in A, provided that we have the SSC provided by Theorem 16.41 In particular, we will 
have either 71-1.2(7) = 1 or 7^2(7) = if 7 ^ A. □ 

10. Remaining Proofs of Theorems in Sj4l We now provide the remaining proofs for four theo- 
rems in 21 At this point, Theorem 14. II has been proved. 

10.1 Proof of Theorem l4.2i We will consider a sequence of stationary Markov processes {{X%(t) : 

t > 0} : n > 1}, with Xq(0) = Xq(oo) for each n > 1. That initial condition makes the stochastic 
processes strictly stationary. We will refer to these stationary processes as stationary versions of the 
processes Xg and denote them by A™. We start by establishing tightness. 

Lemma 10.1 (tightness of the sequence of stationary distributions) The sequences {Ag(oo) : n > 1} and 
{{X"(t) : t > 0} : n > 1} are tight in Rg o,nd T>q, respectively. 

Proof. First, for the tightness of {Xg (00) : n > 1} in Rg, it suffices to treat the six components 
separately. The tightness of {Z"-(oo) : n > 1} in R is immediate because < Z"j(oo) < rrij/n, where 
m™/n — > mj as n — > 00. The tightness of the queue lengths follows from Lemma [731 In particular, since 
(i) Qi(t) <st Ql bd (t) for all * and (ii) Qf(t) (00) and Q£ 6rf (i) Q" M (oo) as t -> 00 for all n > 1, we 

necessarily have Q™(oo) < s t Q" bd (oo) for all n, because stochastic order is preserved under convergence. 
Since Qf bd (oo) => qi,bd(oo) = ?i(0) V {Xi/Oi) as n — > 00, the sequence {Q™ M (oo) : n > 1} is stochastically 
bounded, which implies that the sequence {Q™(oo) : n > 1} is stochastically bounded as well. Since 
tightness of the marginal distributions implies tightness of vectors, the sequence of steady-state random 
vectors {X n (oo) : n > 1} is tight in Rg- Given the tightness of {X™(0)}, the proof of tightness in 2?g is 
identical to the proof of Theorem 15.21 □ 

We next establish the analogue of the structural simplification results in Sj6l Let X(oo) = 
(Qi(oo), 2^(00)) be a limit of the stochastically bounded sequence {X£(oo)}. Let X = (Qi,Zij) be 
a limit of the sequence {{X"(t) : t > 0} : n > 1}. Note that X must itself be a stationary process. 

Lemma 10.2 P(Z 1A (oo) — mi, Z 2 . 1(00) = 0, .£2.2(00) = m 2 — Zi j2 (oo)) = 1. 

Proof. Let /j(oo) = n%j — Zi jfpo) — Z2j(oo), j — 1,2. To prove the claim, we need to show that 
P(Z2,i(oo) > 0) = P(Ij(oo) > 0) = 0. We will consider the sequence of stationary versions {X™}, and a 
limit X of this sequence. 

(i) P(Z2,i = 0e) = 1. We will show that the opposite assumption leads to a contradiction. Hence, 

suppose that P(Z2,i(s) > 0) > for some s > 0. The stationarity of ^2,1 implies that Z 2 ,i(0) = Z2,x(s). 
Hence, we can equivalently assume that P(Z2 i i(0) > 0) > 0. 

Let i?2,i denote the set in the underlying probability space, of all sample paths of X with ^2,1(0) > 0. 
By the contradictory assumption, P(B2.\) > 0. Following the arguments in Lemma [8.11 if Z2,i(0) > 0, 
then Zijt(t) > for all t > 0, which implies that Z 2 ,i{t) > for all f > in i? 2 ,i- 

Consider a sample path in S 2 ,x- Because of one-way sharing, Z\^ — 0e, so that the only departures 
from Qi are due to service completions in pool 1 and abandonment. Since Z\ t \ < ui\ w.p.l, Q\ is 
stochastically bounded from below, in sample-path stochastic order, by the fluid limit of an Erlang-A 
model with m\ servers. At the same time, Q2 is stochastically bounded from above by the fluid limit of 
an Erlang-A model with servers. Hence, there exists e > such that, for some s > 0, 

rQ 2 (t)-e<rq% <q%-K<Qx(t)-K + e for all t>s, (79) 

where rq% < q® — K by Assumption 13 . 1 1 Now, because Q\ and Q 2 are bounded with probability 1, we 
can find sq > such that (|79p holds with this sq for all possible initial condition in B 2 ,x- However, this 
implies that Z 2t \ is strictly decreasing for all t > sq and for all sample paths in B 2 ,i (because no fluid 
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can flow from queue 1 to pool 2), so that ^2,i(i) < ^2,i(so) for all t > sq in £?2,i, contradicting the 
stationarity of J?2,i- Thus, P(£?2,i) = 0. 

(ii) P(Ii = I2 = 0e) = 1. We follow the proof of Theorem 16.21 building on the result P(Z2,i(oo) = 
0) = 1 just established. Recall that L" = Q" + Z™ 1 + Z™ 2 — m ™ m (|5T|) . representing the excess number 
of class-i customers in the system, is stochastically bounded from below, in sample-path stochastic order, 
by the process L b iy with L™ b defined by imposing a reflecting upper barrier at fc™ 2 for i = 1, 2 and letting 
L™ b (0) = Lf(0) A fc™ 2 - However, here we working with stationary versions. Since, the processes (Li,L 2 ) 
are strictly stationary, so are the reflected processes (L™ b ,L 2b ). Then U n > s t U b , where U n and U b and 
the linear functions of (L™,L 2 ) an d (L™ b ,L 2 b ), respectively, defined in (p2")) . These processes U n and U b 
are also stationary processes. 

However, just as in the proof of Theorem 16.21 U b is a BD processes on the integers in (—00, 0], which 
is independent of Z" 2 , with drift S b in (|63|) . Since S b /n — > Sb > 0, the BD U b has a positive drift for 
all n large enough. Consequently, — U b has the structure of the stationary queue length process in a 
stable M/M/l queue. Since the traffic intensity converges to a limit strictly less than one, the initial 
value, and the value at any time, is stochastically bounded. Moreover, we can apply the essentially the 
same extreme value argument used in the proof of Theorem 16.21 to conclude that, for any r > 0, that 
-P(II^HIt > 0) ^ as n ^ 00. We thus conclude that P(||/j|| T = 0) = 1, from which the conclusion 
follows, because the interval [0,oo) can be represented as the countable union of finite intervals of finite 
length, and the countable sum of probabilities is itself zero. □ 

A function Y : M. m — > to, k > 1, is said to be locally Lipschitz continuous if for any compact set B, 
there exists a constant K(B) such that, for any s,t £ B, \Y(s) — Y(t)\ < K{B)\s — 1\. A locally Lipschitz 
continuous function is absolutely continuous and is thus differentiable almost everywhere. 

In the following we will consider two locally Lipschitz continuous functions V : — > R + and Y : 
R+ ->■ R^, and the "Lie derivative" V(Y(t)) = W -Y', where W denotes the gradient of V, and VV -Y' 
is the usual inner product of vectors. Since both functions are differentiable almost everywhere, V(Y(t)) 
is understood to be be taken at points t for which both V and Y are differentiable. 

For a vector x S we let \\x\\ denote its L\ norm, although any other norm in M.^ can be used in 
the following. For the following we draw on §8.3 of [35] . 

Lemma 10.3 Let V : R;t — > M+ be locally Lipschitz continuous, such that V(x) — if and only if x = 0. 
Let Y : R + — > be Lipschitz continuous with constant N , such that Y < M , for some M > 0. Assume 
that V(Y(t)) < for all t > for which Y(t) =/= 0. Then is a globally asymptotically stable point for Y . 

Proof. Let / : M. + — > R + be absolutely continuous, such that f(t) = if and only if t = 0. If for 
almost every t 7^ 0, f'(t) < 0, then f{t) — > as t — > 00. We can apply that argument to the function 
f(t) = V(Y(t)) if we show that V(Y(t)) is locally Lipschitz continuous. To see that is the case, note that 
for all f > 0, 

Y(s) < Y(0) + \Y(s) - Y(0)\ <M + Nt, < s < t. 

Therefore, for any < u < s < t, \f(s) - f(u)\ < K(B)N(s - u), where Be{i£1+: ||x|| < M + Nt}. 
□ 

PROOF of Theorem 14.21 We consider a converging subsequence {X™ : n' > 1} of the sequence 
{X™ : n > 1} with limit X. By Lemma flO. 21 we can consider it to be three-dimensional with components 
Qii Q2 and Zi2- By Corollary 15.11 the limit X is Lipschitz continuous in t, so that it is differentiable 

almost everywhere. Specifically, X is a limit of the sequence represented by (j4Tj) . with ^"(O) = ^"(oo) 
for all n > 1. Then each component of the converging subsequence {X™ } in (|4T|) converges to its 
respective limit (e.g., Q" to Q\). For our purposes here, it is sufficient to conclude that X(t) — > x* w.p.l 
as t — > 00. Hence, we will not characterize the limit of X(t) as t — > 00. 

For the representation of the converging subsequence {X™ } in (|4"Tj) , let 



W n> = (C - l)Zl 2 + CQ\ + Qi, 



: An Averaging Principle 

Mathematics of Operations Research xx(x), pp. xxx-xxx, (c)200x INFORMS 



37 



where C > 1 will be specified later. Since X™ =>■ X by assumption, we can apply the continuous mapping 
theorem to conclude that W n => W in T> 3 as n — > oo, where 

W(t) = (C - l)Zi,a(*) + CQi(<) + Q a (t) 

= (C - l)Zi, 2 (0) - (C^i,2 - Ma.a) / Z h2 (s) - fi 2 , 2 m 2 t 

Jo 

+ CQ 1 (0) + C(\ 1 - I i ltl m l )t-C6 1 [ Qi(s) ds + Q 2 (0) + \ 2 t - 8 2 [ Q 2 (s) ds. 

Jo Jo 

with derivative 

W'(t) = -(GX2 - M2, 2 )^l,2(t) - C0lQl(t) - # 2 Q 2 (<) + C(Ai - + A 2 - ^2,2^2. 

For x e R3, let V(a;) = Cx x + x 2 + (C - l)x 3 and note that V(X) = W ■ X' = W', and that V is 
locally Lipschitz continuous, where W denotes the gradient of V, and W -X' is the usual inner product 
of vectors. Now let Y denote the derivative of X shifted by x* for x* in (JS7J), i.e., Y' = X' + x* . Hence, 
as in the proof of Theorem 8.3 in [35], we have 

V(Y) = -(Cm,2 - M2,2)2i,a(t) - C6xQx{t) - B 2 Q 2 {t). 

If Mi, 2 > A*2,2j then we let C = 1, and if ^1,2 < ^2,2 we let C be any number such that C > ^2,2/^1,2 > 1- 
With this choice of C, we see that V'(Y') < 0. By Lemma 110.31 is globally asymptotically stable 
stationary point for Y, which implies that X(t) — > x* as t — > 00 w.p.l. (Note that Y and X are Lipschitz 
continuous and bounded, as required.) 

For a > 0, let § v {a) = {x e R 3 _ ■ \\V(x) - V(x*)\\ < a}. By_Theorem 10.2 in [35], for each a > 
there exists T(a, X(0)), such that X(t) £ f3v(ct) for all t > T(a, X(0)). Since the queues Q\ and Q 2 are 
bounded w.p.l, we can uniformly bound T(a, X(0)) (uniformly in X(0)). Hence, there exists T = T(a), 
such that X(t) 6 /3y(a) w.p.l for all t > T. It follows from the stationarity of X that X(0) £ /3y(a). 
Since this is true for all a > 0, it must hold that X(0) = x* which, by the equality in distribution 
X(oo) = X(0), implies that X(oo) = x* w.p.l. We have thus shown that the limit of all converging 
subsequences of {X n (oo) : n > 1} is x* in (|27p. which implies the full convergence X n (oo) =3- x* as 

□ 



10.2 Proof of Theorem 14.31 The claimed convergence in D is less complicated than it might 
appear, because there is no spatial scaling. Consequently, all processes are pure-jump processes, having 
piecewise constant sample paths, with only finitely many discontinuities in any bounded interval. Both 
processes have the same four possible transitions from each state: ±1 or ±r for r = r\ i2 . Hence, it suffices 
to show convergence in distribution of the first k pairs of jump times and jump values for any k > 1. 
That can be done by mathematical induction on k. 

We can start by applying the Skorohod representation theorem to replace the assumed convergence in 
distribution T^/n => 76 and D™(T n , 0) D (7, 0) as n — > 00 by convergence w.p.l for alternative random 
vectors having the same distribution. Hence, it suffices to assume that Tg/n — > 76 and £>™(r n ,0) — ► 
-D(7,0) asn->oo and w.p.l. (We do not introduce new notation.) Thus, for each sample point, we will 
have D™(r™,0) = .0(7,0) for all n sufficiently large. Hence, we can assume that we start with equality 
holding. 

Moreover, we can exploit the structure of pure-jump Markov processes. The FTSP is directly such 
a pure-jump Markov process with transition rates given in (Tr2"]) - ([T5")) . These rates were defined to be 
the limit of the transition rates of the queue difference processes D" 2 , after dividing by n. The queue 
difference processes -D™ 2 in Q are not Markov, but they are simple linear functions of the pure-jump 
Markov process X£. 

The transition rates for D\ 2 closely parallel (fT2l) - (fl5| . Because of the assumptions, we can work 
with the three-dimensional random state T n = (Qi,Q 2 ,Zi 2 ) £ K3 with the understanding that the 
remaining components of are Z^ A = m?, Z% A = and Z% 2 = m% - Z^ 2 . Let D n {T n ,t Q ) = D^ 2 (t ) 

under the condition that X n (t Q ) = T n . When O n (F\i ) < 0, let the transition rates be (n, r™), 

X^\n,Y n ), /Lt_ (n.,r n ) and fjP^(n,T n ) in (|4"3")> . for transitions of +r, +1, — r and —1, respectively. When 
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D n (T n ,t ) > 0, let the transition rates be X^\n,T n ), X^\n,T n ), fi^\n,T n ) and ^\n,T n ) in flHJ), for 
transitions of +r, +1, — r and — 1, respectively. 

The many-server heavy-traffic scaling in ([3]) and the condition T n /n — > 7 imply that the transition 
rates of X£ are of order 0(n) as n — > 00. However, the time expansion in (TTT|) brings those transition rates 
back to order 0(1). Indeed, from l[I2 )) -([T5 ]l and (|Ig ]l -(|4"4" |) . we see that the transition rates of D™(F n , •), 
which change with every transition of the CTMC X$, actually converge to the transition rates of the 
FTSP D(j, •), which only depend on the region (D > and D < 0). 

The time until the first transition in the pure jump Markov process D(p/, •) is clearly exponential. Since 
the queue lengths can be regarded as strictly positive by Theorem 16.21 the first transition of D"(r™, •) 
coincides with the first transition time of the underlying CTMC Xq , which also is exponential. Since 
the transition rates converge, the time until the first transition of D™(T n ,-) converges in distribution 
to the exponential time until the first transition of the FTSP -D(7, •). Moreover, in both processes the 
jump takes one of four values ±1 or ±r. The probabilities of these values converges as well. Hence 
the random first pair of jump time and jump value converges to the corresponding pair of the FTSP. 
The same reasoning applies to successive pairs of jump times and jump values, applying mathematical 
induction. That completes the proof. □ 

10.3 Auxiliary Results for FTSP's. Before proving Theorems 14.41 and 14.51 we prove some auxil- 
iary lemmas that we employ in the proofs. We use stochastic bounds by frozen queue-difference stochastic 
processes as in £j T . 1 1 The following lemma is proved much like Lemma IH731 exploiting the bounds in £ )7.2I 

Lemma 10.4 (bounding frozen processes) Suppose that x(t) £ A, t\ < t < t 2 . For any t £ [£1,^2] and 
e > 0, there exist positive constants 8,rj, state vectors x m ,XM € A and random state vectors X^,X^, 
n > 1, such that \\x m — x(t)\\ < e, \\xm — x {~t)\\ < e , n X^ ^ x mi n Xjfa ^ X M as n °°> 

D](X-,-) < r D](X n (t),-) < r H}(Xb,.) and 

D](X™,-) < r Dl 2 (t)< r D](X2 t ,-) in V([t 1 V(t-S),(t + S)At 2 \) for all n>l, (80) 

and P(B n (8,r))) — > 1 as n — > 00, where 

B n (6, n) = {5l{X n M ) < -n, 8 n _{X%) > r,, < -r,, S1(X») > r,} (81) 

As a consequence, the bounding frozen processes D 7 j(X2j,-) and D^lX^,-) in (1801) . and thus also the 
interior frozen processes Dj (X n (t), ■), satisfy (|18[) on B n (5,rf) and are thus positive recurrent there, 
n > 1. 

Proof. We use the convergence X n x provided by Theorem 14. II Consider one t £ [ti,t 2 ]. By the 
linearity of the drift functions 6+ and 5- in CE?]), <5™(X n (f))/n =J> S + (x(t)) and 6 r t{X n (t))/n =*> S-(x(t)) 
for x(t) £ A, so that (fT8|) holds. Hence there exists n > such that 

lim P(5VX n (t)) < -77 and S1(X n {t)) > r?) = 1, 

i.e., ([46]) holds for T" = X n (t) with probability converging to 1 as n — >• 00. 

We now bound the drifts in (|45[) . We do that by bounding the change in the components of X n (t) in 
a short interval around time t. To do that, we use the stochastic-order bounds in Q7.2[ constructed over 
the interval \t\ V (t — 6), (t + S) A t 2 ], with the construction beginning at the left endpoint t\ V (t — 6), just 
as in the construction after time in $7.'2\ Let I/tt,S) = [*i V (£ — 6), (t + £) A t 2 ] and let || • \\s denote the 
norm over the interval 1(^.8)- To construct X^j, let 

= {Ql,Mi Ql,Mi Z M+) an ^ = \Q\,MiQ2,M->Zm-)-: 

where 

Ql M = QIM V 0, Q% M ee \\Q$ tb \\ t , 

Z M+ = ini tei (tl , S ) ^+(*)> z m- = \\ z -\\s, 

with Z\(t) = Z£(t) and Z™(t) = Z£(t) if ^2,2 > /Ui,2, and Z\(t) = Z£(i) and Z™(t) = Z 7 b l (t) otherwise. 
We work with the final value Xl I+ = Xl I+ {{t + S)A t 2 ), and similarly for X^_ . Let {D^X 1 ^, s) : s > 0} 
have the rates determined by X^_ when D^(X^j,s) < 0, and the rates determined by X2 { + when 
D](X^,s)>0. 
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We do a similar construction for X™. Let 

X m+ = \Ql,mi Q%,mi ^m+) an< ^ X m- = {Ql,m> Ql,nn Z m - )i 

where 

= ||Q?, 6 lk Q2, m = inf fe / (fl ,„ Q? l0 (t) v 0, 

with Z£(t) = Z"(t) and ee Z b "(t) if ^2,2 > ^1,2, and Z$(t) = Z£(t) and Z^(i) ee Z%(t) otherwise 

(the reverse of what is done in Xfy). Let {D 7 j(X^, s) : s > 0} have the rates from X^_ when D^(X^, s) < 
0, and the rates from X^ + when Dj(X^, s) > 0. By this construction, we achieve the ordering in (|80l) . 
We cover the rates of D™ 2 (i) too because we can make the identification: the rates of -D" 2 (i) given X ra (i) 
coincide with the rates of D^(X n (t), •). 

It remains to find a <5 such that both the processes {D^X^, s) : s > 0} and {Dj(X^j,s) : s > 0} 
are asymptotically positive recurrent. To do so, we use Lemma 17.31 which concludes that the bounding 
processes as functions of 8 have fluid limits. By Lemma [731 we can conclude that X^ + = n~ 1 X™ i+ =>■ x+ , 

X m- = U ^ 1X n l - X m> X M+ = n ^ X M+ X M and X M~ = n ~ 1X M- Z M in P 3, where IE m+ , X~, 

and a;^ are all continuous with a;+(tiV(t — <5)) = a;~ (ti V(i— 6)) = x^itx V(t— (5)) = £E^(ii V(i— 5)) = 
x(ti V (t — 8)) E A. Hence, we can find <5' such that x m (<5) G A and xm{$) & A for all 8 € [0,(5']. Hence, 
we can choose 8 such that the constant vectors x m = x m (8) and xm = ^m(^) both are arbitrarily close 
to x{t\). 

Finally, we use the linearity of the drift function to deduce the positive recurrence of the processes 
depending upon n. As n — > 00, 

8 n _(X^)/n =* S-(x~), 8l(X- + )/n^8 + (x+), 

8 n _{X n M -)ln =► M*m). and £»(*» + )/n in K. 

□ 

We immediately obtain the following corollary to Lemma 110.41 exploiting Corollary 17.11 

Corollary 10.1 Let £ ee (J V k) — 1 using the QBD representation based on ri, 2 — j/k in §6 of |35j . //, 
in addition to the conditions of Lemma 110.41 

D?(X£,0)-C < si Z?} l (X"(0Vt-,5),0) < st £>y(X^,0) + C ^ 
£>7(X^,0)-C < s * D? i2 (0Vt-5)< si £>7(X^,0) + C in K, n > 1, (82) 
i/ien, m addition to the conclusions of Lemma 110.41 

Dy(X»,t)-C < st D?, 2 (*)< st £>?(*M,<) + C *n ©([iiV^-^Ci + ^Ata]). (83) 

The following lemmas correspond to the QBD structure of the FTSP. These results hold for QBD 
processes much more generally, but we state them in terms of the FTSP. 

Lemma 10.5 (continuity of the stationary distribution of the FTSP) The FTSP stationary random vari- 
able D(7,oo) is continuous in the metric (|84p as a function the state 7 in A and thus D(x(t), 00) is 
continuous in the metric (|84p as a function of the time argument t when x(t) e A. 

Proof. This result follows from Theorem 7.1 in [35], in particular from the stronger differentiability 
of the QBD R matrix in an open neighborhood of each 7 £ A, building on Theorem 2.3 in [16] , The 
second statement follows from the first, together with the continuity of x(t) as a function of t. □ 

Remark 10.1 (continuity on S) Lemma \3 . 1 1 shows that the stationary distribution of the FTSP is well 
defined on all of S provided that we extend the set of possible values of the FTSP to the space E = 
Z U {+00} U {—00}, as in £19.21 Following Theorem 7.1 of [35], we can show that Lemma I10.5I holds, not 
only on A, but also on S, but that extension is not needed here. 

Let r(t) ee inf{w > : D(x(t),u) — s}, where s is a state in the state space of the QBD D. The 
next lemma establishes the existence of a finite moment generating function (mgf) for r(t) for a positive 
recurrent FTSP. 
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Lemma 10.6 (finite mgf for return times) For x(t) G A, let r = r(t) be the return time of the positive 
recurrent QBD D(x(t), •) to a specified state s. Then there exists 9* > such that 4> T (9) = E[e 9l ~] < oo 
for all 6 < 9*. 

Proof. As for any irreducible positive recurrent CTMC, a positive recurrent QBD is regenerative, 
with successive visits to any state constituting an embedded renewal process. As usual for QBD's (see 
[26] ). we can choose to analyze the system directly in continuous time or in discrete time by applying 
uniformization, where we generate all potential transitions from a single Poisson process with a rate 
exceeding the total transition rate out of any state. In continuous time we focus on the interval between 
successive visits to the regenerative state; in discrete time we focus on the number of Poisson transitions 
between successive visits to the regenerative state. 

Let N be the number of Poisson transitions (with specified Poisson rate). The number of transitions, 
N, has the generating function (gf) i/jn(z) = E[z N ], for which there exists a radius of convergence z* 
with < z* < 1 such that tPn{z) < oo for z < z* and ip^(z) = oo for z > z*. 

The mgf 4> T (9) and gf tpi\r(z) can be expressed directly in terms of the finite QBD defining matrices. 
It is easier to do so if we choose a regenerative state, say s* , in the boundary region (corresponding to 
the matrix B in (6.5)-(6.6) of [35]). To illustrate, we discuss the gf. With s* in the boundary level, in 
addition to the transitions within the boundary level and up to the next level from the boundary, we 
only need consider the number of transitions, plus starting and ending states, from any level above the 
boundary down one level. Because of the QBD structure, these key downward first passage times are 
the same for each level above the boundary, and are given by the probabilities Gij[k] and the associated 
matrix generating function G(z) on p. 148 of 26 . Given G(z), it is not difficult to write an expression 
for the generating function ipN n (z), just as in the familiar birth-and-death process case; e.g., see §4.3 of 
[26]. □ 

The next lemmas establish results regarding distances between processes in a discrete state apace. In 
particular, we consider the state-space E of the processes D" 2 an d the FTSP D, which is a countable 
lattice. With the QBD representation (achieved by renaming the states in E) the state-space E is a 
subset of Z. To measure distances between probability distributions on Z, corresponding to convergence 
in distribution, we use the Levy metric, defined for any two cumulative distribution functions (cdf's) F\ 
and F 2 by 

£(F 1 ,F 2 ) = inf{e>0: J F 1 i(a;-e)-e< J F 1 2(a;) < F x (x + e) + e for alia;}. (84) 

For random variables X\ and X 2 , C(Xi, X 2 ) denotes the Levy distance between their probability distri- 
butions. (The Levy metric is defined for probability distributions on R, but we will use it for processes 
defined on the discrete space E.) 

Lemma 10.7 (uniform bounds on the rate of convergence to stationarity) For every to G [ti,t 2 ] there 
exist C > and constants /?o < oo and po > 0, such that 

C(D(x(t), s), D(x(t), oo)) < (3 e~ PoS in R (85) 

holds for all t G [*i V (i - C)> *o + C] C [h , t 2 ] • 

For our proof of Lemma 110.71 we also introduce the concept of uniformly (u, a) — small sets on an 
interval, which generalizes the concept of (u, a)- small sets; for background see, e.g., (53], [3U] and [35]. 
For each (fixed) t > let Pfj(u) denote the transition probabilities of {D(x(t), u) : u> 0}: 

Pl 3 (u) = P(D(x(t),u) — j | D(x(t),0) = i), i, j G Z. 

Definition 10.1 A set C C Z is (u,a)-small, for a time u > and for some a > 0, if there exists a 
probability measure <p (■) on Z satisfying the minorization condition: 

i^.(u) > a^(j), ieC,jGZ. 

We say that a set C is uniformly (u, a) -small in the interval J, if C is (u, a) — small for each process in 
the family {D(x(t) 1 ■) : t G I}, i.e., if C is (u,a)-small for each D(x(t), ■) with the same u and a, t G I. 
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Note that the probability measure tp in the minorization condition is allowed to depend on t in the 
definition of uniformly (u, a) -small. 

Proof of Lemma flO.71 As above, let r(t) denote the return time of the process D(x(t),-) to the 
regeneration state s* , which, for concreteness, we take to be state 0, i.e., s* = 0. Consider the infinitesimal 
generator matrix Q(t) of the process D(x(t),-). In a countable state space, every compact set is small 
(see, e.g., definition in pp. 11 in jM]) and, in particular, {0} is a small set. (We show in (j5T[) below that 
{0} is in fact uniformly (it, a)-small.) Moreover, by Theorem 2.5 in [24], the existence of a finite mgf for 
the hitting time of the small set {0} is equivalent to the existence of a Lyapunov function V : Z — > [1, oo) 
which satisfies the exponential drift condition on the generator, Condition (V4) in |24) : 

Q(t)V < -c t V + d t l {0} , (86) 

where Ct and dt are strictly positive constants. 

Consider the time to <E [ii,^]- Then (|86|) holds at to with constants Ct and dt a . Since Ct > we can 
decrease it such that (|86l) holds with strict inequality and the new ct is still strictly positive. We increase 
dt appropriately, such that J2j Qo,j(t)V(j) < —Ct o V(0) + d to l{ }- The continuity of Q(t) on [ti , £2] as 
a function of t, which follows immediately from the continuity of the rates (|T2"l) - (fT5"l) as functions of the 
continuous function x(t), implies that there exist £ > and two positive constants cq and do, such that 
([55]) holds for all t £ [to — to + C] with the same constants cq and do- However, this is still not sufficient 
to conclude that the bounds in ([55)1 are the same for all t £ [to — (,to + C]j see Theorem 1.1 in [5] (for 
discrete-time Markov chains). 

Recall that for each fixed t, Pf -(s) denotes the transition probabilities of the CTMC {D(x(t), s) : s > 
0}. We can establish uniform bounds on the convergence rates to stationarity by showing that {0} is 
uniformly (u, a) -small in an interval [to — Cj *o + C] I0r the family {D(x(t), ■) : t £ [t — C, £o + C]}> as m 
Definition llO.il In particular, we need to show that 

P '»><V(j), JGZ. (87) 
holds for all t £ [to — (, to + C] with the same a (but is allowed to change with t). This step, together 
with the uniform bounds Co and do in (|5o) established above, will be shown to be sufficient to conclude 
the proof. 

Hence, it is left to show that ((571) holds for all t £ [t — C>*o + C] with the same a > 0. This step is 
easy because {0} is a singleton in a countable state space. Specifically, for each t we consider, we can fix 
any u > and define = PqAu). With this definition of yj* we can take any a < 1 in (|87|) . As in the 

discrete-time case in [5] (the strong aperiodicity condition (A3) in [S] is irrelevant in continuous time), 
the bounds on the convergence rates in (1851) depend explicitly on a in the minorization condition (|87|). 
the bounds in the drift condition (1551) and the Lyapunov function V in ([56]) . This can be justified by 
uniformization, but can also be justified directly for continuous-time processes, e.g., from the expressions 
in Theorem 3 and Corollary 4 in [38 . 

The uniform bounds on the rate of convergence to steady state established above by applying [35J 
are directly expressed in the total- variation metric. If the total variation metric can be made arbitrarily 
small, then so can the Levy metric. Hence we have completed the proof. □ 

Remark 10.2 (bounds on coupling times) The bounds on the rate of convergence to stationarity of 
Markov processes satisfying (l86l) and (|87|) in [38] are obtained via the coupling inequality. In particular, 
we have provided explicit bounds for the time it takes a positive recurrent FTSP Z?(7, •) (with 7 £ A), 
initialized at some finite time, to couple with its stationary version. 

The final auxiliary lemma relates the Levy distance between D™(X n (u),so) in (1TT1) and the FTSP 
D(x(u), so) at a finite time so- 

Lemma 10.8 Suppose that x(t) £ A for t\ < t < £2- Then and for any fixed so > 0, 

£(D2(X n (u), s ), D(x(u), s )) as n — !• oo, uniformly in u £ [£1, £ 2 ]- (88) 

Proof. If follows from the proof of Theorem 14.31 that, for the given e > 0, there exists <5(<i), no{t\) 
and so(^i) such that 

C(D , e l (X n (v),so(t 1 )),D(x(v),so{ti))) <e for all h < v < (h + 6(h)) A t 2 and n > n (ti). 



42 



: An Averaging Principle 
Mathematics of Operations Research xx(x), pp. xxx-xxx, (c)200x INFORMS 



exploiting the convergence X n => x and D n (r n ,0) = £>™ 2 (ti) D(x(t 1 ),0) = D(j,0) for T n = X n {t 1 ) 
and 7 = x(t\). We can apply this reasoning in an open interval about each u € [ti,t 2 ]. In particular, for 
the given e > and u 6 [ti,t 2 ], there exists <5(w), no(u) and so(m) such that 

£(L>; l (X™(u),s (u)),D(a;(w),so(M))) < e for all (u- 5(u)) Vt x < u < (u + S(u)) Ai 2 and n > n (u). 

However, since the interval [ti, t 2 ] is compact and the family of intervals (ti V (u — 5(u), (u + 5(u)) A t 2 ), 
taken to be closed on the left at t\ and closed on the right at f 2 , is an open cover, there is a finite subcover. 
Hence, all time points in [ti,f 2 ] are contained in only finitely many of these intervals. Hence, we can 
achieve the claimed uniformity. Moreover, since the conclusion does not depend on the subsequence used 
at the initial time t\, the overall proof is complete. □ 

10.4 Proofs of Theorems 14.41 and 14.51 We apply the results in ijl0.3l to prove the theorems. 

Proof of Theorem 14.41 First, if x(t ) € A then there exists f 2 such that x{t) £ A over [to, £2] 
because A is an open set and x is continuous. It is possible that the fluid limit never leaves A after time 
to, in which case x € A over [to, 00). In the latter case we can consider any t 2 > to. 

(1) We begin by showing that there exists So = 5 (to) > 0, such that -D™ 2 (ti) is tight in R for all 
ti satisfying to < ti < to + So- Henceforth, t\ denotes such a time point. We need to show that, for 
any e > 0, there exists a constant K such that P(\Df 2 (ti)\ > K) < e for all n > 1. Overall we prove 
the claim in three steps: in the first step we bound D" 2 over [to, to + So], m sample-path stochastic 
order, with positive recurrent QBD's but with random initial conditions. In the second step, we show 
that these bounding QBD's arc tight for each t\ as above, by showing that they couple with stationary 
versions rapidly enough. In the third step we show that we can extend the conclusion from the subinterval 
[ti,to + So] to the entire interval [ti,t 2 ]- 

Step One. We apply Lemma ri0.4l to find a So > and construct random states and X^ with the 
properties stated there, such that rate order holds as in (|50|) in T>([to, to + So]), after we let D^X^, to) = 
D"(X]^-,to) = -D™ 2 (*())■ Next, for all sufficiently large n, we bound the upper bounding process above 
and the lower bounding process below in rate order, each by a FTSP with fixed states x m and xm but 
ordered so that strict rate order in Lemma l7. 41 holds. Since x m and xm are in A and A is open, these new 
states x m and x m can be chosen to be sufficiently near the initial states x m and Xm that they too are 
in A. We let these FTSP's be given the same initial conditions depending on n, consistent with above. 
For that purpose, we put the initial condition in the notation; i.e., we let D(-y, b, •) = D{p/, •) given that 
-D(7,0) = b. We combine the rate order just established with Corollary 1 10. 1 1 to obtain the sample-path 
stochastic order 

D(x m ,Dl 2 (0),nt) - C < sf £>r, 2 (t) < st D{x M ,Dl 2 (0),nt) + C (89) 

for all n > no for some no- 

So far, we have succeeded in bounding D± 2 (t) above (and below) stochastically by a positive recurrent 
FTSP with random initial conditions, in particular starting at D™ 2 (0). It suffices to show that each 
bounding FTSP with these initial conditions in stochastically bounded at any time t\ £ (to, to + So}- We 
show that next. 

Step Two. We now show that the two bounding processes in ([59]) are indeed stochastically bounded 
at such t\. We do that by showing that they couple with the stationary versions of these FTSP's with 
probability converging to 1 as n — > 00. Since x(to) € A, we can apply Theorem 14.11 to deduce that 
X n (to) => x(0) as n — > 00, which implies that 2 (0) = o(n) as n — > 00. If we consider the FTSP 
without the scaling by n in (|89p . it suffices to show, for t\ as above, the bounding processes recover from 
the o(n) initial condition within o(n) time, which will be implied by showing that the coupling takes 
place in time o(n). 

We show that by applying Lemma 110.71 and Remark 110.21 They imply that the time until the FTSP 
in (1891) couples with its stationary version is bounded by constants depending on ct and dt in (|86[) 
and a in ([57]). as well as the Lyapunov function V in (|56"|) . Now, D(xM,to) = Df 2 (to) by construction, 
and x(fo) € A, so that D(xM,to) = o(n). Hence, V(D(xM,t)) — o(n) for all t > t , for V in (|5f)j) . In 
particular, we see that the time until D(xM,ta) couples with it stationary version, when initialized at 
D i 2 (to), is o(n). Together with (|89|) . that implies the claim. That is, we have shown that D" 2 (ti) is 
tight in R for all ti, to < ti < to + So- 
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Step Three. To extend the result to the interval [i 1 , £2] we observe that we can repeat the reasoning 
above for any starting point t € [t\, £2] (since x(t) £ A for all t G [£1, £2]), achieving an uncountably- infinite 
cover for [£1, £2] of intervals of the form [t, t + 8(t)]. Since the interval [£1, £2] is compact, the uncountably 
infinite cover of these intervals, made open at the left unless the left endpoint is £1 and made open on the 
right unless the right endpoint is £2, has a finite subcover. As a consequence, the entire interval [£i,£2] is 
covered by only finitely many of these constructions, and we can work with the finite collection of closures 
of these intervals. 

In particular, since the sequence {D r { 2 (t\) : n > 1} is stochastically bounded, we can apply the 
construction over an interval of the form [£i,£'] for t\ < £' < £2. As a consequence we obtain stochastic 
boundedness for each £ in [t\ , t'\ . If £' < £2, then we continue. We then can choose a second interval 
[£",£"'] such that t x < t" < t' < t'" . We thus can carry out the construction over [£",£"']. Since t" < t', 
we already know that the sequence of random variables {D\ l 2 (£"); n > 1} is stochastically bounded from 
the first step. Thus, in finitely many steps, we will deduce the first conclusion in (i). If x(0) € A and we 
take to = 0, then D" 2 (to) is tight in R because D\ l 2 (0) =>■ L by Assumption 13.21 so the result holds on 
[£ ,£ 2 ] = [0,£ 2 ]. 

(ii) To prove the statement in (ii) we observe that Theorem 14.31 implies that the oscillations are 
asymptotically too rapid for the sequence {D\ l 2 : n > 1} to be tight in T> over any finite interval. 
Moreover, there is a finite interval over which a single frozen-difference process serves as a lower bound, 
by Lemma 110.41 Because of the scaling by n, the maximum in the lower bound QBD over this interval 
will be unbounded above (actually of order O(logn) by reasoning as in Lemma 17.61) . Thus, the sequence 
of stochastic processes {D r { 2 '■ n > 1} is not even stochastically bounded over any finite subinterval of 

[*o,*a]. 

(Hi) We apply Lemma 171)1 to establish (|30[) . We can bound the supremum of D™ 2 (t) over the interval 
[£i,£2] by the supremum of the finitely many frozen queue-difference processes. Since the rates are of 
order n, Lemma [7.61 implies (l30l) . Once again, the result can be extended to hold on [£o,£2] = [0,£ 2 ] if 
x(0) £ A by the assumed convergence D™ 2 (0) => L in Assumption 13.21 □ 

Proof of Theorem 14.51 For any given t with t\ < t < t 2 and e > 0, we will show that we can 
choose tiq such that £(D™ 2 (£), D(x(t), 00)) < e for all n > jiq. Since {_D™ 2 (£i) : n > 1} is stochastically 
bounded, it is tight. Hence, we start with a converging subsequence, without introducing subsequence 
notation. The result will not depend on the particular converging subsequence we choose. 

Hence, we start with D™ 2 (t\) ^> L, where L is a proper random variable. We will then let D(x(ti), 0) 
/ to obtain D" 2 (ti) =>■ D(x(ti), 0), as required to apply Theorem 14.31 at £1. If t > tx, then for all 
sufficiently large n, t — (s/n) > ti, in which case we can write D\ l 2 (t) = D" 2 (t — (s/n) + (s/n)) = 
D™ (X n (t— (s / n) , s), where -D™ is the expanded queue-difference process defined in (TTT1) . with t—(s/n) > t\ 
so that x(t — (s/n)) £ A. Hence we can write 

C(Dl 2 (t),D(x(t),^)) < C(D n e (X n (t-( S /n), S ),D(x(t - (s/n)),s)) 

+C(D(x(t - (s/n)),s),D(x(t - (s/n)), 00)) + C(D(x(t ~ (s/n)), 00), D(x(t), 00)). (90) 

For t and any e > given, we first choose d so that t — S > t\ and x remains in A throughout [0, t + S], 
which is always possible because x(t) G A, £ > £1, x is continuous and A is an open subset of S. (We use 
the condition that t > t\ here.) In addition, we choose 5 sufficiently small that the third term in (l90|) is 
bounded as 

C(D(x(u), 00), D(x(t), 00) < e/3 for all u, t-5<u<t + 5, 

which is possible by virtue of Lemma 110.51 Given t, e and 8, we choose sq sufficiently large that the 
second term in (|90l) is bounded as 

C(D(x(u), so), D(x(u), 00) < e/3 for all u, t — S<u<t + S, 

which is possible by Lemma [10.71 Finally, we choose no sufficiently large that sa/n < 5 for all n > no 
and the first term in (|90|) is bounded as 

£(D"(X n (u,s ),D(x(u),so)) < e/3 for all u, t-5<u<t + 5, 



which is possible by Lemma |10. 81 That choice of S, sq and no makes each term in (|90)) less than or equal 
to e/3 for all n > n , thus completing the proof. □ 
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11. Conclusion and Further Research. In this paper we proved a FWLLN (Theorem 14. ip for 
an overloaded X model operating under the fixed-queue-ratio- with-thresholds (FQR-T) control ( Ej2.2p . 
with many-server heavy-traffic scaling ( H2.3p . Theorem 14.11 shows that the fluid-scaled version of the 
six-dimensional Markov chain Xq, whose sample-path representation appears in Theorem 15.11 converges 
to a deterministic limit, characterized by the unique solution to the three-dimensional ODE (|22p . which 
in turn is driven by the fast-time-scale stochastic process (FTSP) in H3.2I We also proved a WLLN for 
the stationary distributions (Theorem l4.2l) that justifies a limit interchange in great generality (Corollary 



Finally, in i)4.3l wc presented results regarding the queue difference process and SSC for the queues 
when the fluid limit is in A. in particular, Theorem 14.41 proved statements regarding the tightness of 
D " 2 (t) in R, and non-tightness in T>. Corollary 14.21 shows that SSC for the queues hold under any scaling 
larger than 0(log n), depending only whether the fluid limit x is in A. Theorem l4.5l establishes a pointwise 
AP result, which is not an immediate corollary of the AP in the FWLLN. 

Proof of the FWLLN. We proved the FWLLN in three steps, showing that: (i) the sequence of pro- 
cesses {Xq : n > 1} is tight and every limit is continuous (Theorem 15.2)1 : (ii) simplifying the rep- 
resentation in Theorem 15.11 to the essentially three-dimensional representation in fill) (Corollary 16. 4p : 
and (iii) characterizing the limit via the averaging principle (AP) (<JHJ). Characterizing the fluid limit 
of that three-dimensional process was challenging because the sequence of queue-difference processes 
{{D™ 2(i) : t > 0} : n > 1} in ^ does not converge to any limiting process asn-} 00. Instead, we have 
the AP, which can be better understood through Theorems 14.3114.51 

Due to the AP, the indicator functions 1{d^ 2 (s)>o} an d 1{d™,(s)<o} m the- representation (|4ip are 
replaced in the limit with appropriate steady-state quantities related to the FTSP D(x(t),-), e.g., 
l{z>™ 2 (s)>0} is replaced in the limit with 



Our proof of the AP in Sj9]is based on the framework established by Kurtz in [25 . In the appendix 
we present a different proof of the AP which is weaker, since it only characterizes the FWLLN in the set 
A. However, it has the advantage of being intuitive, with an explicit demonstration of the separation of 
time scales which takes place in A. Moreover, this proof continues the bounding logic of §Sj7]and[51 and 
thus follows naturally from previous results established in the paper. Both proofs have merits, and can 
be useful in other models where separation of time scales occur in the limit. 

Results for QBD Processes. In the process of proving the SSC in Theorem 16.41 and Theorems 14.41 
and 14.51 we established some general results for QBD's, which are interesting in their own right. First, 
we established an extreme- value result in Lemma 17.61 Then, in H10.3I we established the continuity of 
stationary distributions (Lemma 110.51) . finite mgf for return times (Lemma ll0.6p and uniform ergodicity 
for a family of ergodic QBD processes (Lemma ll0.7p . To the best of our knowledge, those results are not 
stated in the existing literature. 

The Control. The FQR-T control is appealing, not only because it is optimal during the overload 
incident in the fluid limit [53], but also because the FQR-T control produces significant simplification of 
the limit, since it produces a strong form of state space collapse (SSC). Specifically, by Theorem l6.41 the 
four-dimensional service process is asymptotically one dimensional, with Z™ 2 alone characterizing limits 
under any scaling. By Theorem l4.41 the two-dimensional queue process is asymptotically one dimensional 
under any scaling larger than O(logn) (in particular, under fluid and diffusion scaling). This latter result 
holds unless class 1 is so overloaded, that there is not enough service capacity in both pools to keep 
the desired ratio between the two queues (in which case the fluid limit will not be in A). Hence, the 
six-dimensional Markov chain describing the system during overloads in the prelimit, is replaced by a 
simplified lower dimensional deterministic function, which is easier to analyze. In addition, this SSC 
results are crucial to proving the FCLT for the system )3"o] . 

Further Research. As Lemma 18.11 and the proof of Theorem 16.41 reveal, one-way sharing, which we 
suggested in )33j as a means to prevent unwanted simultaneous sharing of customers in finite systems 
(where the thresholds themselves may not be sufficient to prevent two-way sharing) has shortcomings; 
see Remark |8. II For example, in the limiting fluid model, if after some time the overload switches over, 
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so that queue 2 should start receiving help from pool 1, then the one-way sharing rule will not allow 
class-2 customers to be sent to pool 1 since, as was shown in the proof of Theorem 16. 4[ zi^{t) > for all 
t > 0. As a consequence, in large systems, a significant amount of time must pass before sharing in the 
opposite direction is allowed. This problem with one-way sharing can be remedied by dropping that rule 
completely, or by introducing lower thresholds on the service process; again see Remark 18.11 We have 
begun studying such alternative controls. 

Appendix A. Alternative Proof of Characterization Using Stochastic Order Bounds. In 

this appendix we present an alternative proof of Lemma l9~2l and thus an alternative proof of the FWLLN 
in Theorem 14.11 At the beginning of $9] we observed that it suffices to characterize the limiting integral 
terms in (|68l) ; i.e., it suffices to prove Lemma \§7Z[ In SjHJwe accomplished that goal by using the martingale 
argument of [T71[S5]. Here we show that same goal can be achieved with stochastic bounds, exploiting 
Lemma 110.41 and similar reasoning. However, we prove less than the full Theorem 14.11 here. Our proof 
here is under the special case of Assumption 13.21 for which x(0) € A. For this special case we carry out 
the characterization over the full interval [0, co) if x(t) remains within A for all t > 0. Otherwise, we 
complete the characterization proof over [0, T^], where 

T A = inf {t> : x(t) £ A}. (91) 

Since x is continuous and A is an open subset of S, we know that Ta > 0. A first step is to do the 
characterization over an interval [0, 8] for some 8 > 0. We start in EIA.1I by indicating how the interval 
of convergence can be extended given that the first step has been carried out. Next in EIA.2I we prove 
Theorem 14.11 subject to Lemma [A.21 establishing convergence of integral terms over the interval [0,(5]. 
For that purpose, we state Lemma IA.4I establishing a sample path stochastic order bound that we will 
use to prove Lemma IA.2I in ^A.3I we establish continuity results for QBD processes. We then proves 
Lemmas I A . 2 1 and I A . 41 respectively, in E1A.4I and $A.5\ 

A.l Extending the Interval of Convergence. Unlike the first proof, with this second proof we 
only establish convergence in X>i4([0, Ta]) if Ta < oo, for Ta in (|9"Tj) . We now show how we achieve this 
extension. 

As in the first proof, after establishing the convergence over an initial interval [0,(5] with 8 < r, we 
apply Theorem [Hil] to conclude that any limit point of the tight sequence X$ is again a limit of the tight 
sequence Ag'* in (J9j> over the entire half line [0,oo), showing that r places no constraint on expanding 
the convergence interval. Moreover, by part (ii) of Theorem 5.2 in [35], any solution to the ODE, with a 
specified initial condition, can be extended indefinitely, and is unique. Hence that places no constraint 
either. 

However, for this second method of proof, we do critically use that fact that x(t) S A, < t < 8, in 
order to prove the characterization. (This is proved in Lemma I A. 1 1 below. ) However, we can extend the 
interval of convergence further. Given that we have shown that X(t) = x{t) £ A for A in (fT5|) over a time 
interval [0, 8], and thus established the desired convergence X n x over that time interval [0, <5], we can 
always extend the time interval to a larger interval [0,(5'] for some 8' with 8' > 8. To do so, we repeat 
the previous argument treating time 8 as the new time origin. That directly yields X{t) = x(t) G A, 
and thus convergence X n =>■ x, over the time interval [(5,(5']. However, we can combine that with the 
previous result to obtain X(t) — x(t) £ A, and thus convergence X n x, over the longer time interval 
[0,5']. Let v be the supremum of all 8 for which the expansion of convergence to [0,(5] is valid. We 
must have X(t) = x{t) £ A, and thus convergence X n =>- x, over the interval [0,^), open on the right. 
The interval is [0,oo) if v = oo. Suppose that v < oo. In that case, we can next apply continuity 
to extend the interval of convergence to the closed interval [0,i/]. Since X(t) = x(t), < t < v : x is 
continuous and X is almost surely continuous, we necessarily have X{v) = x(v) w.p.l as well. We claim 
that v > Ta = inf{£ > : x{t) ^ A}. If not, we can do a new construction yielding X(t) — x(t), first for 
v < t < v 1 and then for < t < v 1 , v 1 > v, contradicting the definition of v. Hence, we have extended 
the domain of convergence to [0, T A ] if Ta < oo and to [0, oo) otherwise, as claimed. 

A. 2 Reduction to Convergence of Integral Terms. Since each of these integrals in can be 
treated in essentially the same way, we henceforth focus only on the single term I q ,i y \{t) and establish 
(EH). 
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Recall that X is the limit of the converging subsequence of {X n : n > 1} in ©aQO, r]) for r in Theorem 
16.31 An important first step is to identify an initial interval [0,<5], < 5 < r, over which X 6 A. We 
apply the proof of Lemma 110.41 to prove the following lemma. 

Lemma A.l (state in A over [0,<5]) There exists S with < <5 < r for r in Theorem \6 .31 such that 

P(X n (t) e A, < t < S) -> 1 as n ->• 00, 
/or A in CEH]), so ttai P(X(t) G A < t < S) = 1. 

Proof. Recall that we have assumed that Assumption 13.21 holds with x(0) G A. We can apply 
the first step of the proof of Lemma 110.41 to obtain the stochastic bound over an initial interval [0, 6] 
for some 5 > 0. Since the FTSP £>(7, •) is positive recurrent if and only if 7 € A for A in (fT5)) . By 
Lemma [10.41 Df(X™ l7 -) and Df(X 7 ^ I ,-) are positive recurrent on B n (S,n) in (I5T1) . Thus, by Corollary 
110.11 Df(X n (t), •) is positive recurrent on B n (8,rj) as well for < t < 6. Hence, the claim holds. □ 



The next important step is the following lemma, proved in £1A.4[ after establishing preliminary bound- 
ing lemmas. 



Lemma A. 2 (convergence of the integral terms) There exists 6 with < S < t for r in Theorem \ 6.3[ 
such that for any e > and t with < t < 5, there exists a = a(e, 5, t) with < < 5 — t and no such 
that ^ 

P (\\J t 1 {Di 2 ( s )>o}Zt 2 (s) ds-7r h2 (X(t))Z 1 , 2 (t)\>e\ <e for all n>n . (92) 

In order to apply Lemma lA. 21 to prove Lemma 19.21 we exploit the absolute continuity of Iq,x,i, estab- 
lished now. 

Lemma A. 3 (absolute continuity of Iq 1,1) The limiting integral term almost surely satisfies 

< I q ,i,i(t + u) - Iq,i,i(t) < m 2 u for all <t <t + u < t, (93) 

and so Iq t \.\ is the cumulative distribution function corresponding to a finite measure, having a density 
h depending on X . As a consequence, for all e > 0, there exists uq > such that 



I q ,iAt + u) - I q , 1A (t) _ 



< e (94) 



for all u < Uq. 
Proof. Since 



ft+u 

Iq,i,x{t + u) -q tlil (t) = I ^{D l2( s)>o}Zl 2 (s)ds < um^/n, 



Iqi 1W i s a nondecreasing function with < P q \ x (t + u) — I ql x (t) < nij for all < t < t + u < r. 
Hence, I ql j(t) is a a cumulative distribution function associated with a finite measure. The convergence 
obtained along the subsequence based on tightness then yields 

< I q ,i, i(t + u) - Iq,x,i(t) < m 2 u for all 0<t<t + u<T. 

Hence, has a density with respect to Lebesgue measure, as claimed. □ 

Proof of Lemma 19.21 Given Lemma IA.21 for any e > 0, we can find a and no such that (|9"2")l is 
valid for all n > Uq. Hence, we can let n — > 00 and conclude that, for any e > 0, we can find a such that 

P + a ) - - ^,2(A(t))Z lj2 (t)| > ^ < e. (95) 

However, given that (|94|) and (|95|) both hold, we conclude that we must almost surely have h(t) — 
7i"i,2(A(i))Zi. 2 (t), which completes the proof. □ 

We now apply bounds to prove Lemma IA.21 The comparisons in Lemma 110.41 and Corollary 110.11 are 
important, but they are not directly adequate for our purpose. The sample-path stochastic order bound 
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in Corollary 1 1 . 1 1 enables us to prove Lemma [A. 21 and thus Theorem 14. 1[ for the special case of ri 2 = 1, 
because then £ = 0, where £ is the gap in Corollarv llO.il but not more generally when the gap is positive. 
However, we now show that an actual gap will only be present rarely, if we choose the interval length 
small enough and n big enough. We use the construction in the proof of Lemma 110.41 exploiting the 
fact that we have rate order, where the bounding rates can be made arbitrarily close to each other by 
choosing the interval length suitably small. We again construct a sample path stochastic order, but for 
the time averages of the time spent above 0. We prove the following lemma in ^A.5l We use it to prove 
Lemma [PI in 3A.4I 

Lemma A. 4 Suppose that conditions of Lemma 110.41 hold. Then, for any e > 0, there exists no and 
£ with < £ < S for S in Lemma 110.41 such that, in addition to the conclusions of Lemma 110.41 and 
Corollary 110.11 the states x m ,XM € A and random vectors X^,X^ 4 and associated frozen processes 
Z?^(X") and Dy(XjJf) can be chosen so that on B n (^,rj) (defined as in (1811) with £ instead of 8 and 
necessarily P(B n (£, n)) — > 1 as n — > oo) 7 first 

£>7(X£, 0) = D?(X&, 0) = DIM (96) 

and, second, 

t l 1 ^ D /( X "' s)>0 > dS ~ 6 ~ st 1 J 1 { £, ?,aW>°} ds -s< 1 1 {DJ(XS I ,s)>0} ds + e ( 97 ) 

for n > no and < t < f; i.e., there is sample path stochastic order in 2?([0, £]) for n > no- 

A. 3 Continuity of the FTSP QBD. In the current proof of Lemma |9~21 and thus Theorem 14. II we 
will ultimately reduce everything down to the behavior of the FTSP D. First, we intend to analyze the 
inhomogeneous queue-difference processes D™ 2 (r n ) in ([6]) in terms of associated frozen processes D 1 j{T n ) 
introduced in ^7.11 obtained by freezing the transition rates at the transition rates in the initial state 
T". In (|47|) above, we showed that the frozen-difference processes can be represented directly in terms 
of the FTSP, by transforming the model parameters (Xi,rrij) and the fixed initial state 7 and scaling 
time. We will appropriately bound the queue-difference processes D n (T n ) above and below by associated 
frozen-queue difference processes, and then transform them into versions of the FTSP D. For the rest of 
the proof of Theorem 14. II in Sj5] wc will exploit a continuity property possessed by this family of pure-jump 
Markov processes, which exploits their representation as QBD processes using the construction in §6 of 
|35) . We will be applying this continuity property to the FTSP D. 

To set the stage, we review basic properties of the QBD process. We refer to §6 of [35] for important 
details. From the transition rates defined in p^|) - ([T5|) . we see that there are only 8 different transition 
rates overall. The generator Q (in (65) of [35]) is based on the four basic 2m x 2m matrices B, Aq, A\, and 
A2, involving the 8 transition rates (as shown in (66) of [35]). By Theorem 6.4.1 and Lemma 6.4.3 of [26) . 
when the QBD is positive recurrent, the FTSP steady-state probability vector has the matrix-geometric 
form a n = aoR n , where a n and czo are 1 x 2m probability vectors and R is the 2m x 2m rate matrix, 
which is the minimal nonnegative solutions to the quadratic matrix equation Aq + RA\ + R 2 A2 — 0, and 
can be found efficiently by existing algorithms, as in |26j ; See [35) for applications in our settings. If the 
drift condition (ITS]) holds, then the spectral radius of R is strictly less than 1 and the QBD is positive 
recurrent (Corollary 6.2.4 of [26]). As a consequence, we have X^^Lo = ~ R)^ 1 ■ Also, by Lemma 
6.3.1 of [26) . the boundary probability vector cto is the unique solution to the system ao(B + RA2) = 
and al — ao(I — i?) _1 l = 1. See §6.4 of [3S] for explicit expressions for 711,2(7) f° r 7 G A. 

As in Lemma 110.61 we also use the return time to a fixed state, r, and its mgf <fi T (6) with a critical 
value 9* > such that <fi T (8) < 00 for 6 < 8* and (j> T {6) = 00 for 9 > 9* . We will be interested in the 
cumulative process 

C(t)= [ (f(D(s))-E[f(D(oo))])ds t>0, (98) 
Jo 

for the special function f(x) = l{a;>o}. Cumulative processes associated with regenerative processes 
obey CLT's and FCLT's, depending upon assumptions about the basic cycle random variables r and 
J Q T f(D(s)) ds, where we assume for this definition that -D(O) = s*; see §VI.3 of [5] and [TJ]. From [7], 
we have the following CLT with a Berry-Esseen bound on the rate of convergence (stated in continuous 
time, unlike [7j): For any bounded measurable function g, there exists to such that 

\E[g{C{t))/Vt] - E[g(N(Q,a 2 ))}\ < for all t > t , (99) 
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where 

2 



a 2 = E 



f(D(s))-E[f(D(oo))]ds 



(100) 



again assuming for this definition that D(0) = s*. The constant K depends on the function g (as well as 
the function / in (|98|l) and the third absolute moments of the basic cycle variables defined above, plus 
the first moments of the corresponding cycle variables in the initial cycle if the process does not start in 
the chosen regenerative state. 



There is significant simplification in our case, because the function / in (|98[) is an indicator function. 
Hence, we have the simple domination: 

\f(D(s)\ds= f f(D(s))ds<T w.p.l (101) 
o Jo 

As a consequence, boundedness of absolute moments of both cycle variables reduces to the moments of 
the return times themselves, which are controlled by the mgf. 

We will exploit the following continuity result for QBD's, which parallels previous continuity results 
for Markov processes, e.g., [21] |4T|. 

Lemma A. 5 (continuity of QBD's) Consider a sequence of irreducible, positive recurrent QBD's having 
the structure of the fundamental QBD associated with the FTSP in ^3 - 21 and §6 in [35 with generator 
matrices {Q n : n > 1} of the same form. If Q n —¥ Q as n —¥ oo [which is determined by the convergence 
of the 8 parameters) , where the positive-recurrence drift condition (|18l) holds for Q, then there exists no 
such that the positive-recurrence drift condition (|18[) holds for Q n for n > uq. For n > no, the quantities 
(R, ao, a, </> r , O*-, ipNi z * \ c 2 j K) indexed by n are well defined for Q n , where a 2 and K are given in (|99[) and 
(|100[) . and converge as n — > oo to the corresponding quantities associated with the QBD with generator 
matrix Q. 

Proof. First, continuity of R, ao and a follows from the stronger differentiability in an open 
neighborhood of any 7 € A, which was shown to hold in the proof of Theorem 5.1 in [35], building on 
Theorem 2.3 in [16]. The continuity of a 2 follows from the explicit representation in (|100|) above (which 
corresponds to the solution of Poisson's equation). We use the QBD structure to show that the basic cycle 
variables r and J Q f(D(s)) ds are continuous function of Q, in the sense of convergence in distributions 
(or convergence of mgf 's and gf 's) and then for convergence of all desired moments, exploiting (|101j) and 
the mgf of t to get the required uniform integrability. Finally, we get the continuity of K from [7J and 
the continuity of the third absolute moments of the basic cycle variables, again exploiting the uniform 
integrability. We will have convergence of the characteristic functions used in [7J. However, we do not 
get an explicit expression for the constants K . □ 

We use the continuity of the steady-state distribution a in <|A4l specifically in (|109[) . In addition, we 
use the following corollary to Lemma IA.5I in (|108[) in flA.4l We use the notation in f|47jl . 

Corollary A.l If (\™,fh",%) — > (Xi,rrij,j) as n — > 00 for our FTSP QBD's, where jTS} holds for 
(Xi, mj, j), then for all e > there exist t Q and n such that 

P (\\ J 1 {D(\™, m ™, 7n ,s)>o}ds~ P(L»(A. i ,m J ,7, 00) > 0)| > < e 
for all t > to and n > no . 

Proof. First apply Lemma IA.5I for the steady-state probability vector a, to find no such that 
|P(D(A", to", 7„, 00) > 0)| — P(D(Xi, mj, 7, 00) > 0)| < e/2 for all n > n . By the triangle inequality, 
henceforth it suffices to work with P(D(A™, to™, 7„, 00) > 0) in place of P(D(Xi,mj,j ) 00) > 0) in the 
statement to be proved. By (|99]) . for any M, there exists to such that for all t > to, 

(It / 1 {n(A",m", 7 „, s )>o} ds - P(Z)(A",TO™,7 n ,oo) > 0)| > -j= ) 
\tJo VtJ (1Q2) 

J ftTA™,TO™,7„) 

<P(|A(0,a 2 (A",TO", 7n ))| >M)+ ^' 3 ' 7 ' 
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We get (fTU2|) from flM|) by letting f(x) = l( ,oo)(^) in ® and letting g(x) = l{\ x \ >M }{x) in 

Next, choose M so that P(\N(0, a 2 (Xi, rrij, 7))! > M) < e/2. Then, invoking Lemma [A.51 increase 
no and to if necessary so that |ct 2 (A™, m™, 7„)) — a 2 (X i7 mj, -f))\ and \K(Xf, to™, 7„) — K(X i7 mj, -f)\ are 
sufficiently small so that the right side of (|102[) is less than e/2 for all n > uq and t > to- If necessary, 
increase to and no so that M/^/to < e/2. With those choices, the objective is achieved. □ 

A. 4 Proof of Lemma IA.2i In this subsection we give the first of two long proofs of previous 
lemmas. We now prove Lemma IA.21 which completes the alternate proof of Theorem 14.11 in the case 
7*1,2 = 1, because then Corollary 1 1 . 1 1 holds with gap £ = 0. In that case, Corollary 110.11 directly implies 
that Lemma [A. 41 holds with e = 0. Lemma IA.4| which is proved in the following subsection, is required 
to treat more general r\ y i. 

First, let S > 0, e > and t with < t < 6 be given, where the S is chosen as in Lemma [10.41 (with 
5 < t for r in Theorem I6.3p . Here we will be introducing a new interval [t, t + a], where < a = a(t) = 
a(t, e, 5) < 5 — t, so that [t, t + a] C [0, 5}. Moreover, we will make a < £ for £ existing via Lemma [A.4I 
Lemmas 110.41 and IA.4I hold on the interval [t,t + £], where £ = satisfies < £ < S — t. We will be 
choosing a with < a < £. 



Before we started with regularity conditions at time provided by Assumption 13.21 We now will exploit 
tightness to get corresponding regularity conditions at time t here. In particular, before we started with 
the convergence X n (0) =>- x(0) in R 3 where x(0) G A based on Assumption 13.21 Now, instead, we 
base the convergence X n (t) X(t) at time t on the convergence established along the converging 
subsequence (without introducing new subsequence notation) . We apply Lemma IA.1I to deduce that 
P(X(t) G A) = 1. Since the frozen processes to be constructed are Markov processes, we can construct the 
processes after time t, given only the value of X n (t), independently of what happens on [0, t]. Before, we 
also started with D™ 2 (0) L, where L is a finite random variable. Instead, here we rely on the stochastic 
boundedness (and thus tightness) of {D™ 2 (t) : n > 1} in K provided by Theorem 14.41 As a consequence, 
the sequence {(X n (t), D™ 2 (t)) : n > 1} is tight in M 2 - Thus there exists a convergent subsequence of 
the latest subsequence we are considering. Hence, without introducing subsequence notation, we have 
(X n (t),D? 2 (i)) ^> (X(t),L(t)) in R 2 as n -> 00, where (X(t),L(t)) is a finite random vector with 
P(X(t) G A) = 1. 

We use the same construction used previously in the proofs of Lemmas 110.41 and lA.4[ letting a decrease 
to achieve new requirements in addition to the conclusions deduced before. We now regard t as the time 
origin for the frozen difference processes. Given Di 2 (t), let the two associated frozen difference processes 
be {D n f {Xlj, s):s>t} and {DJ(X%t, s) : s > 0}.'We directly let 

DJiX^t) = DJiX^t) = D](X n (u),t) = Dl 2 (t), u>t, (103) 

so that we can invoke property (|96p at time t in our application of Lemma lA. 41 here. As before, the initial 
random states X^ and X^ and their fluid-scaled limits are chosen to achieve the goals before and here. 

We now successively decrease upper bounds on a and increase lower bounds on n until we achieve 
(|92|) in Lemma TA.21 First, we can apply Lemma [10.41 to find an ni such that P(B n (5,e) > 1 — e/6 for 
n > n%. We next apply Lemma [A. 41 to conclude that there exists <j\ such that the following variants of 
the integral inequalities in ([9"T|) hold with probability at least 1 — e/6 as well: 

~/ hD 7( x z , s) >o } ds--^-<^J t l W , 2 ( s) >o } ds<~^ MD 7 (x^s ) >o } ds + -^-. (104) 

for all a < <T\ (We divide by vn 2 because we will be multiplying by zi^ 2 (t).) 

We now present results only for Xjfa, with the understanding that corresponding results hold for X™ n . 
We represent the bounding frozen queue-difference processes directly in terms of the FTSP, using the 
relation (|47|) . with the notation introduced there: 

{X>y(A?,m?,Xjk,i + *) : s > 0} = {D(X? /n,m] /n, X2Jn,t + sn) : s > 0}. (105) 
Upon making a change of variables, the bounding integrals in (|104l) become 



-I 



d 1 



l{£)]?(AT',my,X£,s)>0} ds — / l{D(\™/n,mpn,X"-°/n,s)>0}ds. (106) 
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For each integer fc, we can apply Lemma [A. 51 to obtain the iterated limits 

lim lim P{D{X} /n,m% /n,X^/n,s) = k) = lim lim P(D(\?/n, m?/n, X^/n, s) = fc), (107) 



where the limit is P(D(xm, oo) = fc) = P(D(Xi,rrij,XM,oo) = k). In particular, the limit for the model 
parameters first implies convergence of the generators. The convergence of the generators then implies 
convergence of the processes. Finally Lemma IA.5I implies convergence of the associated steady-state 
distributions. 

By Corollary IA.11 we also have the associated double limit for the averages over intervals of length 
0(n) asn-> oo. As n — > oo, 



1 



no 



t+na 



/ ^{D(XV/n,m^/n,X" /n,s)>0}ds ^ P(D(Xi,rnj,XM, oo) > 0) =Tri,2(x M )- (108) 



Invoking Lemma |A.5[ choose a 2 less than or equal to the previous value o\ such that 

ki, 3 (a; ro ) - TT h2 (X(t))\ < for all a < a 2 . (109) 

07712 

For that a 2 , applying (|108[) . choose n 2 > n\ such that for all n> n 2 , 

( 1 rt+n^2 e \ e 

P Ife It 1 { D (K/n,m 7 /n,x- I /^s)>o} ds - n h2 (x M )\ > < g- ( U °) 

We now use the convergence of X n along the subsequence over [0, t) together with the tightness of the 
sequence of processes {X n : n > 1} to control Z" 2 in an interval after time t. In particular, there exists 
03 < o 2 and 773 > n 2 such that 

P{ sup {\X n {u) - X(t)\} > e/6) < e/6 for all n > 7i 3 . (Ill) 

u:t<u<t-\-(T3 

For the current proof, we will use the consequence 

P{ sup {\Z£ 2 (u) - Z h2 (t)\} > e/6) < e/6 for all n > 77 3 . (112) 

u:7<u<7+<73 

We let the final value of a be 03. We now show the consequences of the selections above. We will directly 
consider only the upper bound; the reasoning for the lower bound is essentially the same. Without loss of 
generality, we take e < 1 A 7772. From above, we have the following relations (explained afterwards) holding 
with probability at least 1 — e (counting e/6 once each to achieve \\xm —X (£) || < e/6, \\x m — X(t)\\ < e/6, 
(fTMl) . (ITT21 and twice for (fTTOl) ): 

(a) / l{Dr, 2 (s)>o}Z" 2 (s) d s < (Zi,2(t) + |) / l{£>r j2 ( a )>o} ^ 



(6) < (\ 2 (*) + l{Dy(A ? ,my,XJi,a)>0} ds + £ 

(c) = (^l, 2 (i) + (J 1{D{\™ /n, m pn,Xl,/n,t+sn)>n} ds + 

(d) = (^l,a(*) + gj ( — J l{D(Ar/n,»r.7/n,X»/n,t+*)>0} ds + — 



(e) < (Zi, a (t) + |) tr ^1,2 (xm) + ^) 

(/) < (^> 2 (*) + |) ff (^™) + ^) 
G?) < Z x , 2 (t)-K 1}2 (X{t))o + ^S^ eg + lea 



67772 

2e \ " (H3) 



6 2 12m 2 

(A) <Z lt2 (t)7T 1}2 (X(t))a+^ea 

<{Zi, 2 {t)TTi, 2 {X{t)) + t)a for all 77 > n Q = n 3 . 



: An Averaging Principle 

Mathematics of Operations Research xx(x), pp. xxx-xxx, (c)200x INFORMS 



51 



We now explain the steps in (|113|) : First, for (a) we replace Z™ 2 (s) by Zi^if) for t < s < t + £ by 
applying (|112l) . For (b), we apply Lemma lA. 41 For (c), we use the alternative representation in terms of 
the FTSP in (|105l) . For (d), we use the change of variables in f| 106|) . For (e), we use (|110[) . exploiting the 
convergence in (|108|) . For (f), we use (|109[) . Step (g) is simple algebra, exploiting Z\${t) < m 2 . Step (h) 
is more algebra, exploiting ni : 2(X(t)) < 1, and e < 1 A m^. That completes the proof of the lemma. □ 



A. 5 Proof of Lemma IA.4i We will directly show how to carry out the construction of the sample 
path order for stochastic processes having the same distributions as the terms in (|97p . We will show that 
the interval [0,£] can be divided into a large number (0(n)) of alternating subintervals, each of length 
0(l/n), such that full sample path order holds on one subinterval, and then the processes are unrelated 
on the next subinterval. We will then show, for an appropriate choice of £, that these intervals can be 
chosen so that the first intervals where order holds are much longer than the second intervals where the 
processes are unrelated. Hence wc will deduce for this construction that the inequalities in ([§?]) holds 
w.p.l for all n sufficiently large. We will specify the steps in the construction and explain why they 
achieve their goal, while minimizing the introduction of new notation. We highlight twelve key steps in 
the argument. 

An integer state space. We exploit Assumption 13.31 to have r% t 2 be rational, and then use the integer 
state space associated with the QBD representation in §6 of [35] ■ However, we do not directly exploit the 
QBD structure; instead we directly work with the CTMC's with transitions ±j and ±fc, where j and k 
are positive integers. We will reduce the analysis to consideration of homogeneous CTMC's. 

Exploiting Lemma 110. 41 but reducing the length of the interval. We construct the states x m ,XM G A 
and random vectors X^, X^ and associated frozen processes D^X^) and DJ^X^) exactly as in Lemma 
110.41 but we adjust the specific values to make them closer together at time as needed by choosing £ 
to be suitably smaller than the <5 needed in Lemma 110.41 It is significant that here £ can be arbitrarily 
small; we only require that £ > 0. 

Initializing at time 0. By Assumption 13.21 we have X n =4> x(0) £ A C M 3 and £>™ 2 (0) 4>Lel. 
Since x(0) is deterministic, we immediately have the joint convergence (X n ,D" 2 (0)) =>■ (x(0),L) £ 
R4; apply Theorem 11.4.5 of [45]. We then apply the Skorohod representation theorem to replace the 
convergence in distribution with convergence w.p.l., without changing the notation. Hence, we can start 
with (X n ,D r { 2 (Q)) (x{0),L) £ R 4 w.p.l. With that framework, we condition on D% 2 (0). We then 
initialize the processes D](X™,0) and D^Xj^O) to satisfy D^(X™,0) = D^(X^,0) = £>? 2 (0), as in 

Coupling over an initial random interval. Given identical initial values, we can apply the rate order 
in Lemma 110.41 to construct versions of the processes that will be ordered w.p.l over an initial interval 
of random positive length, just as in Corollarv llO.il Let v n be the first time that the sample path order 
is violated. The rate order implies that v n > w.p.l for each n > 1. The coupling is performed over 
the interval [0, v n \. At random time v n , we must have all three processes in the boundary set of states 
{j : —j3 + 1 < j < /?}, where /3 = j V k. That is so, because violation of order only need occur when, 
just prior to the order violation, we have —ft < DJ(X^) < < 1 < D^X^) < /?, where the rates are 
no longer ordered properly, because the processes are in different regions. At time z/„, either the upper 
process jumps down below the lower process or the lower process jumps up over the upper process. 

A new construction using independent versions when order is first violated. At time v n , the order is 
first violated and would remain violated over an interval thereafter. However, at this time v n , we alter 
the construction. At this random time v n we temporarily abandon the coupling based on rate order. 
Instead, going forward from time v n , we construct independent versions of the three processes being 
considered. More precisely, the three processes are conditionally independent, given their initial values 
at time v n . The idea is to let them evolve in this independent manner until the three processes reach 
a state in which the desired sample path ordering does again hold. We do this in a simple controlled 
manner. We wait until, simultaneously, the upper bound process Dj(X2 { , v n +t) exceeds a suitably high 
threshold, the lower bound process D 7 j(X^, v n + 1) falls below a suitably low threshold, and the interior 
processes Df ^i^n + 1) is in a middle region. At such a random (stopping) time, the three processes will 
necessarily be ordered in the desired way. After that time, we can use the coupling again. 
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Avoiding working directly with D™ 2 . However, since D™ 2 (v n + t) is an inhomogeneous CTMC, and 
thus difficult to work with, we avoid working with it directly. Instead, we simultaneously construct 
new upper and lower bound frozen processes, D^Xjfa) and -Dj(X^) starting at time u n , coupled with 
W.2K +t) : t > 0}, initialized by stipulating that DJ{X^,u n ) = D](X^,v n ) = D^ 2 (u n ). These 
new processes are coupled with {D™ 2 (i/ n + 1) : t > 0}, but conditionally independent of the independent 
versions of the other two processes {DJ (X^ , v n +t) : t > 0} and {D^X^, v n +t) : t > 0}. We again can 
apply Lemma 110.41 to obtain rate order, but again we cannot have full sample path order, because of the 
gap. Nevertheless, we can now apply Corollary 1 10. II to conclude that we have the sample path stochastic 
order 

vn +t)-0 < st Dl 2 (y n + t) < st D](Xlr, v n + t) + p in D([u n , 5}). 
We thus can do a sample path construction to achieve 

b n f (X^u n +t)-P<Dl 2 {v n +t)<D r ;(X^v n +t)+l3 for < t < 5 - u n w.p.l. (114) 

Stopping times for desired order to be achieved. We now define stopping times for the order to be 
achieved. Let 

<A„ = inf {t > : D](Xh, v n +t)> 2/3, D](X£, v n + t) < -2/3, -/3 < D n f (X^,v n + t) < f3}, (115) 

with the three processes (X™) and D^X^.j) in (I115P being mutually conditionally indepen- 

dent, given that all three are initialized at time v n with the final values there obtained from the evolution 
of the coupled processes up until time v n . (We have conditional independence given the vector of initial 
values at time v n .) We can combine (I115P with (|114l) to conclude that we have the appropriate order at 
time v n + <p n , in particular, 

D](X^, v n + <f> n ) < Dl 2 (v n + <t> n ) < D](X2t, v n + n ) w.p.l. 

We not only obtain the desired order at time v n + <j) n , but the stopping time <p n depends on the three 
processes {D](X^,v n + t) : t > 0}, {D n f {X , 1 ! n ,v n + t) : t > 0} and {D^X^, v n + t) : t > 0}, which are are 
independent homogeneous CTMC's conditional on their random state vectors and X^, respectively. 
Thus the stopping time is with respect to the three-dimensional vector- valued CTMC (conditional on the 
random state vectors X^ and X"J. 

Alternating Cycles using coupled and independent versions. Going forward, after the time v n -\- cf) n for 
4> n in (|115[) . we again construct coupled processes just as we did at time 0. However, now we initialize 
by having the specified states coincide at time v n + </>„; i.e., D^(X^,v n + <p n ) = D n ^{X^v n + <f> n ) = 
Di 2 {v n + 4> n ). Hence, we can repeat the coupling done over the initial interval [0, v n \. In particular, we 
can apply the ordering of the initial states and the rate ordering to construct new coupled versions after 
time v n + 4> n - We can thus repeat the construction already done over the time interval [0, v n \ after time 
v n + 4> n ■ The coupled construction beginning at time v n + </>„ will end at the first subsequent time that 
the order is violated. At this new random time (paralleling v n ), we must have all three processes in the 
boundary set of states {j : —j3 + 1 < j < /3}. In this way, we produce a sequence of alternating intervals, 
where we perform coupling on one interval and then create independent versions on the next interval. 

Applying a FWLLN after scaling time by n. Since the transition rates of D™ 2 and all the other 
processes are 0(n), there necessarily will be order 0(n) of these cycles in any interval [0, £]. However, we 
scale time by n, replacing t by nt, just as in (|4"T)) . (I105[) and (|106[) to represent all processes as FTSP's 
with random parameters depending on n, converging to finite deterministic limits. Once, we scale time 
by n in that way, the scaled transition rates converge to finite limits as n — ¥ 00, but the relevant time 
interval becomes [0, n£] instead of [0, £], as in (|106p . We can thus apply a FWLLN to complete the proof. 

Regenerative Structure as a basis for the FWLLN. As a formal basis for the FWLLN, we can apply 
regenerative structure associated with successive epochs of order violation starting from coupling, but a 
regenerative cycle must contain more than two successive intervals; we do not have an alternating renewal 
process. Such order- violation epochs necessarily occur in the boundary region {j : —j3 + 1 < j < /?}, 
where (3 = j V k. Hence, there are at most (2/3) 3 vectors of values for the three processes. Moreover, 
there necessarily will be one of these vectors visited infinitely often. Successive visits to that particular 
state vector after coupling thus serve as regeneration points for the entire process. That is, there is an 
embedded delayed renewal process. We next ensure that the times between successive regeneration times 
have finite mean values, with the correct asymptotic properties as n — > 00. That justifies the FWLLN. In 
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particular, we can apply a FWLLN for cumulative processes, as in [T^]. For the particular QBD processes 
being considered, there is continuity of the asymptotic parameters, as indicated in Lemma lA.51 

Finite mean time between regenerations of order 0(l/n). From basic CTMC theory, it follows that 
the first passage times 4> n in (|115[) have finite mean values which are of order 0(1/ n). In particular, 
E[(j) n ] < Cn,\ < oo, where nc n ^\ —¥ ci > 0. It is more difficult to treat the mean time over which the order 
is valid during a single coupling, i.e., the initial time v n and the subsequent random times that which 
the order is valid during the coupling. However, we can truncate the variables at finite constant times in 
order to ensure that the FWLLN reasoning can be applied; e.g., we replace v n by v n = v n A c n ^ < oo, 
where nc n ^ — > C2 > 0. We can later choose ci large enough so that the inequalities in (p?T|) hold for 
the specified e. Finally, the total number of these cycles until the designated initial state vector appears 
again is a random variable. Since the successive vector of initial state vectors visited on successive cycles 
is a finite-state discrete-time Markov chain, the random number of cycles within a regeneration interval 
is a random variable with finite mean, say c n ,3, with c n ^ — > C3 > as n — ¥ 00. As a consequence, the 
mean time between successive regenerations is c n .4 < 00, where nc„.4 — > C4 > as n — > 00. 

The proportion of time that order holds. Finally, it is important that we can control the proportion 
of time that order holds in any interval [0,t], < t < £, as stated in (|97|) . for an appropriate choice 
of £. As a basis for that control, we exploit the fact that the distribution of </> n , the length of the first 
interval during which order violation is allowed, and the random lengths of all subsequent intervals on 
which the desired sample path order is violated, depend on the states of the three processes at time v n , 



i.e., upon D n f (X™ 



and D n _Ay n 



However, because the violation necessarily occurs in 
the boundary region {j : —f3 + 1 < j < (3}, where (3 = j V k, there are at most 2(3 possible values for 
each process, and thus at most (2/3) 3 vectors of values for the four processes. Since this number is finite, 
the violation interval lengths (like <f> n ) can be stochastically bounded above, independent of the specific 
values at the violation point. Moreover, this can be done essentially independently of £. On the other 
hand, the length of the intervals on which the coupling construction remains valid necessarily increases 
as we decrease £ and appropriately increase C2 and c„.2 in the paragraph above. Thus, for any e > 0, we 
can achieve the claimed ordering in (|97[) for all n > no by an appropriate choice of £, C2 and ng. □ 



Appendix B. The Bounding QBD in Lemma l8.3l We now describe how to present the process 
-D™ in Step One of the proof of Lemma l8.3l as a QBD for each n. We assume that ra 1 = j/k for j, k £ Z+, 
and let m = j V k. We define the process D™ = kQ^^ — jQi t *- Thus, _D™ is £>" with an altered state 
space. In particular, each process is positive recurrent if and only if the other one is. 

We divide the state space Z + = {0, 1,2,...} into level of size m: Denoting level i by L(i), we have 

L(0) = (0,1,..., m- 1), L(l) = (m,m + l,...,2m- 1) etc. 

The states in L(0) are called the boundary states. Then the generator matrix Q^ n ' of the process D" has 
the QBD form 



Q (n) = 





An) 








4 Tl) 


4"' 


An) 












An 








An) 
A 2 


4" 



V 



/ 



(All matrices are functions of A"™. However, to simplify notation, we drop the argument A™, and similarly 
in the example below.) 

For example, if j = 2 and k — 3, then 
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AS 





A 2 ' 
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-cr" 
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-cr" 
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)■ 


A 2 ' 





-cr" 
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where £g = fq + pQ and <r n = /xg + A™ + A3 1 . 

is an irreducible CTMC infinitesimal generator matrix. It is 
easy to see that its unique stationary probability vector, i», is the uniform probability vector, attaching 
probability 1/m to each of the m states. Then by Theorem 7.2.3 in [26], the QBD is positive recurrent 
if and only if isA^l < vA^f 1 !, where 1 is the vector of all l's. This translates to the stability condition 
given in the proof of Lemma 18.31 
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